[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126328808105-19048-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:12:09 dist-test-slave-gkw7 krb5kdc[19057](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:12:09 dist-test-slave-gkw7 krb5kdc[19057](info): set up 2 sockets
Aug 07 18:12:09 dist-test-slave-gkw7 krb5kdc[19057](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260807 18:12:10.422349 19048 mini_kdc.cc:121] Time spent starting KDC: real 1.454s user 0.000s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
W20260807 18:12:10.653247 19048 mini_kdc.cc:253] Time spent creating user principal joe-interloper: real 0.126s user 0.001s sys 0.005s
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126328808105-19048-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126328808105-19048-0/krb5kdc/test-user.keytab.
W20260807 18:12:11.103056 19048 mini_kdc.cc:302] Time spent creating keytab for test-user: real 0.450s user 0.003s sys 0.003s
Aug 07 18:12:11 dist-test-slave-gkw7 krb5kdc[19057](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126331, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:12:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:12:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.154.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.154.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.154.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.62.keytab.
Entry for principal kudu/127.18.154.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.62.keytab.
W20260807 18:12:11.640987 19048 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.18.154.62: real 0.479s user 0.000s sys 0.011s
I20260807 18:12:11.643074 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126328808105-19048-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126328808105-19048-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126328808105-19048-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.154.62:40979
--webserver_interface=127.18.154.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.62.keytab
--principal=kudu/127.18.154.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.154.20:40709
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.154.62:40979
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126328808105-19048-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126328808105-19048-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126328808105-19048-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:12:12.037151 19073 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:12:12.037681 19073 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:12:12.037868 19073 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:12:12.045444 19073 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:12:12.045666 19073 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:12:12.045784 19073 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:12:12.045910 19073 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:12:12.046007 19073 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:12:12.056766 19073 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.154.20:40709
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126328808105-19048-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126328808105-19048-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.154.62:40979
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.62.keytab
--principal=kudu/127.18.154.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.154.62:40979
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.154.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126328808105-19048-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:12:12.059010 19073 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:12:12.060958 19073 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:12:12.084622 19082 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:12:12.085070 19081 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:12:12.085626 19086 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:12:12.086509 19073 server_base.cc:1034] running on GCE node
I20260807 18:12:12.087769 19073 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:12:12.090103 19073 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:12:12.091363 19073 hybrid_clock.cc:648] HybridClock initialized: now 1786126332091292 us; error 92 us; skew 500 ppm
I20260807 18:12:12.091979 19073 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:12:12 dist-test-slave-gkw7 krb5kdc[19057](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126332, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:12:12.100972 19073 init.cc:377] Logged in from keytab as kudu/127.18.154.62@KRBTEST.COM (short username kudu)
I20260807 18:12:12.103734 19073 webserver.cc:460] Webserver started at http://127.18.154.62:41215/ using document root <none> and password file <none>
I20260807 18:12:12.104928 19073 fs_manager.cc:359] Metadata directory not provided
I20260807 18:12:12.105144 19073 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:12:12.105731 19073 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:12:12.111301 19073 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126328808105-19048-0/minicluster-data/master-0/data/instance:
uuid: "2ccef3794e914916b5b983bf5bd893a0"
format_stamp: "Formatted at 2026-08-07 18:12:12 on dist-test-slave-gkw7"
server_key: "f607347d1819fea11bef9ccd0dc2943a"
server_key_iv: "907a2ef529e2275d275f024671db84ff"
server_key_version: "encryptionkey@0"
I20260807 18:12:12.112488 19073 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126328808105-19048-0/minicluster-data/master-0/wal/instance:
uuid: "2ccef3794e914916b5b983bf5bd893a0"
format_stamp: "Formatted at 2026-08-07 18:12:12 on dist-test-slave-gkw7"
server_key: "f607347d1819fea11bef9ccd0dc2943a"
server_key_iv: "907a2ef529e2275d275f024671db84ff"
server_key_version: "encryptionkey@0"
I20260807 18:12:12.121124 19073 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.010s sys 0.000s
I20260807 18:12:12.126829 19094 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:12:12.128732 19073 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.004s sys 0.000s
I20260807 18:12:12.129029 19073 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126328808105-19048-0/minicluster-data/master-0/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126328808105-19048-0/minicluster-data/master-0/wal
uuid: "2ccef3794e914916b5b983bf5bd893a0"
format_stamp: "Formatted at 2026-08-07 18:12:12 on dist-test-slave-gkw7"
server_key: "f607347d1819fea11bef9ccd0dc2943a"
server_key_iv: "907a2ef529e2275d275f024671db84ff"
server_key_version: "encryptionkey@0"
I20260807 18:12:12.129284 19073 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126328808105-19048-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126328808105-19048-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126328808105-19048-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:12:12.154385 19073 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:12:12.160466 19073 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:12:12.160910 19073 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:12:12.186470 19073 rpc_server.cc:307] RPC server started. Bound to: 127.18.154.62:40979
I20260807 18:12:12.186503 19168 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.154.62:40979 every 8 connection(s)
I20260807 18:12:12.188810 19073 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
I20260807 18:12:12.196121 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 19073
I20260807 18:12:12.196604 19048 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126328808105-19048-0/minicluster-data/master-0/wal/instance
I20260807 18:12:12.197296 19170 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:12:12.200321 19048 external_mini_cluster.cc:1428] Setting key dc2d1e573233d48b31c5b6e727e8be10
I20260807 18:12:12.224849 19170 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ccef3794e914916b5b983bf5bd893a0: Bootstrap starting.
I20260807 18:12:12.232856 19170 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2ccef3794e914916b5b983bf5bd893a0: Neither blocks nor log segments found. Creating new log.
I20260807 18:12:12.235294 19170 log.cc:824] T 00000000000000000000000000000000 P 2ccef3794e914916b5b983bf5bd893a0: Log is configured to *not* fsync() on all Append() calls
I20260807 18:12:12.240983 19170 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ccef3794e914916b5b983bf5bd893a0: No bootstrap required, opened a new log
Aug 07 18:12:12 dist-test-slave-gkw7 krb5kdc[19057](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126331, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.154.62@KRBTEST.COM
I20260807 18:12:12.254375 19170 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2ccef3794e914916b5b983bf5bd893a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ccef3794e914916b5b983bf5bd893a0" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 40979 } }
I20260807 18:12:12.254941 19170 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2ccef3794e914916b5b983bf5bd893a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:12:12.255172 19170 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2ccef3794e914916b5b983bf5bd893a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ccef3794e914916b5b983bf5bd893a0, State: Initialized, Role: FOLLOWER
I20260807 18:12:12.256318 19170 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2ccef3794e914916b5b983bf5bd893a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ccef3794e914916b5b983bf5bd893a0" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 40979 } }
I20260807 18:12:12.256670 19170 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2ccef3794e914916b5b983bf5bd893a0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:12:12.257009 19170 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2ccef3794e914916b5b983bf5bd893a0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:12:12.257334 19170 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2ccef3794e914916b5b983bf5bd893a0 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:12:12.260888 19170 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2ccef3794e914916b5b983bf5bd893a0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ccef3794e914916b5b983bf5bd893a0" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 40979 } }
I20260807 18:12:12.262305 19170 leader_election.cc:302] T 00000000000000000000000000000000 P 2ccef3794e914916b5b983bf5bd893a0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2ccef3794e914916b5b983bf5bd893a0; no voters:
I20260807 18:12:12.262931 19173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:12.210686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34690 (local address 127.18.154.62:40979)
0807 18:12:12.211926 (+ 1240us) server_negotiation.cc:206] Beginning negotiation
0807 18:12:12.211938 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 18:12:12.211988 (+ 50us) server_negotiation.cc:407] Connection header received
0807 18:12:12.228801 (+ 16813us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:12:12.228825 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:12:12.229410 (+ 585us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:12:12.229933 (+ 523us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:12:12.236817 (+ 6884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:12.238858 (+ 2041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:12.242527 (+ 3669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:12.243348 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:12.243580 (+ 232us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:12.251993 (+ 8413us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:12:12.252071 (+ 78us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:12:12.252107 (+ 36us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:12:12.252162 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:12:12.255957 (+ 3795us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:12:12.257016 (+ 1059us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:12:12.257027 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:12:12.257038 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:12:12.257346 (+ 308us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:12:12.257900 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:12:12.257905 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:12:12.257908 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:12:12.258883 (+ 975us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:12:12.259331 (+ 448us) server_negotiation.cc:1035] Waiting for connection context
0807 18:12:12.260241 (+ 910us) server_negotiation.cc:299] Negotiation successful
0807 18:12:12.260683 (+ 442us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":892,"thread_start_us":598,"threads_started":1}
I20260807 18:12:12.263445 19170 leader_election.cc:288] T 00000000000000000000000000000000 P 2ccef3794e914916b5b983bf5bd893a0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:12:12.263795 19176 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2ccef3794e914916b5b983bf5bd893a0 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:12:12.266790 19170 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2ccef3794e914916b5b983bf5bd893a0 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:12:12.268313 19176 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2ccef3794e914916b5b983bf5bd893a0 [term 1 LEADER]: Becoming Leader. State: Replica: 2ccef3794e914916b5b983bf5bd893a0, State: Running, Role: LEADER
I20260807 18:12:12.269893 19176 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2ccef3794e914916b5b983bf5bd893a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ccef3794e914916b5b983bf5bd893a0" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 40979 } }
I20260807 18:12:12.278409 19177 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ccef3794e914916b5b983bf5bd893a0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2ccef3794e914916b5b983bf5bd893a0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ccef3794e914916b5b983bf5bd893a0" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 40979 } } }
I20260807 18:12:12.279008 19177 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ccef3794e914916b5b983bf5bd893a0 [sys.catalog]: This master's current role is: LEADER
I20260807 18:12:12.279759 19178 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ccef3794e914916b5b983bf5bd893a0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2ccef3794e914916b5b983bf5bd893a0. Latest consensus state: current_term: 1 leader_uuid: "2ccef3794e914916b5b983bf5bd893a0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ccef3794e914916b5b983bf5bd893a0" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 40979 } } }
I20260807 18:12:12.280212 19178 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ccef3794e914916b5b983bf5bd893a0 [sys.catalog]: This master's current role is: LEADER
I20260807 18:12:12.283424 19193 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:12:12.292088 19193 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:12:12.306908 19193 catalog_manager.cc:1347] Generated new cluster ID: 3c53e55b80f5463499a58d027380f2e6
I20260807 18:12:12.307071 19193 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:12:12.324419 19193 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:12:12.326229 19193 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:12:12.348737 19193 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2ccef3794e914916b5b983bf5bd893a0: Generated new TSK 0
I20260807 18:12:12.350011 19193 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.154.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.154.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.154.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.1.keytab.
Entry for principal kudu/127.18.154.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.1.keytab.
I20260807 18:12:12.410295 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126328808105-19048-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126328808105-19048-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126328808105-19048-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126328808105-19048-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.154.1:0
--local_ip_for_outbound_sockets=127.18.154.1
--webserver_interface=127.18.154.1
--webserver_port=0
--tserver_master_addrs=127.18.154.62:40979
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.1.keytab
--principal=kudu/127.18.154.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.154.20:40709
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126328808105-19048-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126328808105-19048-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126328808105-19048-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:12:12.783077 19207 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:12:12.783366 19207 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:12:12.783528 19207 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:12:12.791361 19207 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:12:12.791472 19207 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:12:12.791697 19207 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.154.1
I20260807 18:12:12.802433 19207 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.154.20:40709
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126328808105-19048-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126328808105-19048-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.1.keytab
--principal=kudu/127.18.154.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.154.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126328808105-19048-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.154.1
--webserver_port=0
--tserver_master_addrs=127.18.154.62:40979
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.154.1
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126328808105-19048-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:12:12.803838 19207 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:12:12.805253 19207 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:12:12.820806 19213 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:12:12.821120 19214 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:12:12.821153 19216 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:12:12.822266 19207 server_base.cc:1034] running on GCE node
I20260807 18:12:12.822964 19207 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:12:12.824043 19207 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:12:12.825284 19207 hybrid_clock.cc:648] HybridClock initialized: now 1786126332825244 us; error 55 us; skew 500 ppm
I20260807 18:12:12.825723 19207 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:12:12 dist-test-slave-gkw7 krb5kdc[19057](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126332, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:12:12.829298 19207 init.cc:377] Logged in from keytab as kudu/127.18.154.1@KRBTEST.COM (short username kudu)
I20260807 18:12:12.831154 19207 webserver.cc:460] Webserver started at http://127.18.154.1:38115/ using document root <none> and password file <none>
I20260807 18:12:12.831956 19207 fs_manager.cc:359] Metadata directory not provided
I20260807 18:12:12.832111 19207 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:12:12.832468 19207 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:12:12.835729 19207 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126328808105-19048-0/minicluster-data/ts-0/data/instance:
uuid: "3d370252c4a84a758df83d5ac74a59df"
format_stamp: "Formatted at 2026-08-07 18:12:12 on dist-test-slave-gkw7"
server_key: "fbe0db7ab5709af51965150a2e052540"
server_key_iv: "646b63d3662ed15590b48a6870b05212"
server_key_version: "encryptionkey@0"
I20260807 18:12:12.836654 19207 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126328808105-19048-0/minicluster-data/ts-0/wal/instance:
uuid: "3d370252c4a84a758df83d5ac74a59df"
format_stamp: "Formatted at 2026-08-07 18:12:12 on dist-test-slave-gkw7"
server_key: "fbe0db7ab5709af51965150a2e052540"
server_key_iv: "646b63d3662ed15590b48a6870b05212"
server_key_version: "encryptionkey@0"
I20260807 18:12:12.842378 19207 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260807 18:12:12.845993 19225 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:12:12.847416 19207 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 18:12:12.847621 19207 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126328808105-19048-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126328808105-19048-0/minicluster-data/ts-0/wal
uuid: "3d370252c4a84a758df83d5ac74a59df"
format_stamp: "Formatted at 2026-08-07 18:12:12 on dist-test-slave-gkw7"
server_key: "fbe0db7ab5709af51965150a2e052540"
server_key_iv: "646b63d3662ed15590b48a6870b05212"
server_key_version: "encryptionkey@0"
I20260807 18:12:12.847841 19207 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126328808105-19048-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126328808105-19048-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126328808105-19048-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:12:12.876509 19207 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:12:12.881470 19207 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:12:12.881837 19207 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:12:12.883046 19207 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:12:12.884768 19207 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:12:12.884891 19207 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:12:12.885113 19207 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:12:12.885203 19207 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:12:12.931766 19207 rpc_server.cc:307] RPC server started. Bound to: 127.18.154.1:41259
I20260807 18:12:12.931864 19375 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.154.1:41259 every 8 connection(s)
I20260807 18:12:12.933168 19207 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126328808105-19048-0/minicluster-data/ts-0/data/info.pb
I20260807 18:12:12.943101 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 19207
I20260807 18:12:12.943357 19048 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126328808105-19048-0/minicluster-data/ts-0/wal/instance
I20260807 18:12:12.944105 19048 external_mini_cluster.cc:1428] Setting key d1caf1509f5ab0df334f3f20042f0f6a
Aug 07 18:12:12 dist-test-slave-gkw7 krb5kdc[19057](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126332, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.1@KRBTEST.COM for kudu/127.18.154.62@KRBTEST.COM
I20260807 18:12:12.958695 19380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:12.937111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.1:33841 (local address 127.18.154.62:40979)
0807 18:12:12.937700 (+ 589us) server_negotiation.cc:206] Beginning negotiation
0807 18:12:12.937705 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:12:12.938729 (+ 1024us) server_negotiation.cc:407] Connection header received
0807 18:12:12.940832 (+ 2103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:12:12.940838 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:12:12.940941 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:12:12.941072 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:12:12.943143 (+ 2071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:12.943989 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:12.946452 (+ 2463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:12.946959 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:12.947062 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:12.952236 (+ 5174us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:12:12.952241 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:12:12.952245 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:12:12.952277 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:12:12.955044 (+ 2767us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:12:12.955785 (+ 741us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:12:12.955795 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:12:12.955797 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:12:12.955925 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:12:12.956491 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:12:12.956500 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:12:12.956507 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:12:12.956971 (+ 464us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:12:12.957119 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0807 18:12:12.958226 (+ 1107us) server_negotiation.cc:299] Negotiation successful
0807 18:12:12.958413 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":469,"thread_start_us":319,"threads_started":1}
I20260807 18:12:12.960301 19381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:12.937681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:40979 (local address 127.18.154.1:33841)
0807 18:12:12.938535 (+ 854us) negotiation.cc:107] Waiting for socket to connect
0807 18:12:12.938587 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0807 18:12:12.940539 (+ 1952us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:12:12.941370 (+ 831us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:12:12.941378 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:12:12.942032 (+ 654us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:12:12.942912 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:12:12.942944 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:12:12.944184 (+ 1240us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:12:12.944190 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:12:12.946308 (+ 2118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:12:12.946332 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:12:12.947295 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:12:12.947305 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:12:12.947691 (+ 386us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:12.948480 (+ 789us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:12:12.948521 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:12:12.952002 (+ 3481us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:12:12.955189 (+ 3187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:12:12.955199 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:12:12.955215 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:12:12.955645 (+ 430us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:12:12.956042 (+ 397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:12:12.956047 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:12:12.956051 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:12:12.956289 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:12:12.957121 (+ 832us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:12:12.957132 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:12:12.957670 (+ 538us) client_negotiation.cc:769] Sending connection context
0807 18:12:12.958068 (+ 398us) client_negotiation.cc:240] Negotiation successful
0807 18:12:12.958730 (+ 662us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":614,"thread_start_us":381,"threads_started":1}
I20260807 18:12:12.962232 19376 heartbeater.cc:347] Connected to a master server at 127.18.154.62:40979
I20260807 18:12:12.962838 19376 heartbeater.cc:464] Registering TS with master...
I20260807 18:12:12.963775 19376 heartbeater.cc:511] Master 127.18.154.62:40979 requested a full tablet report, sending...
I20260807 18:12:12.968377 19120 ts_manager.cc:194] Registered new tserver with Master: 3d370252c4a84a758df83d5ac74a59df (127.18.154.1:41259)
WARNING: no policy specified for kudu/127.18.154.2@KRBTEST.COM; defaulting to no policy
I20260807 18:12:12.970858 19120 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.154.1@KRBTEST.COM'} at 127.18.154.1:33841
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.154.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.154.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.2.keytab.
Entry for principal kudu/127.18.154.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.2.keytab.
I20260807 18:12:13.001765 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126328808105-19048-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126328808105-19048-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126328808105-19048-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126328808105-19048-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.154.2:0
--local_ip_for_outbound_sockets=127.18.154.2
--webserver_interface=127.18.154.2
--webserver_port=0
--tserver_master_addrs=127.18.154.62:40979
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.2.keytab
--principal=kudu/127.18.154.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.154.20:40709
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126328808105-19048-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126328808105-19048-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126328808105-19048-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:12:13.368932 19386 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:12:13.369246 19386 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:12:13.369464 19386 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:12:13.378610 19386 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:12:13.378734 19386 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:12:13.378985 19386 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.154.2
I20260807 18:12:13.390445 19386 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.154.20:40709
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126328808105-19048-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126328808105-19048-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.2.keytab
--principal=kudu/127.18.154.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.154.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126328808105-19048-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.154.2
--webserver_port=0
--tserver_master_addrs=127.18.154.62:40979
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.154.2
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126328808105-19048-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:12:13.391940 19386 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:12:13.393469 19386 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:12:13.409178 19396 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:12:13.409267 19392 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:12:13.409266 19393 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:12:13.410666 19386 server_base.cc:1034] running on GCE node
I20260807 18:12:13.411502 19386 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:12:13.412539 19386 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:12:13.413777 19386 hybrid_clock.cc:648] HybridClock initialized: now 1786126333413708 us; error 68 us; skew 500 ppm
I20260807 18:12:13.414113 19386 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:12:13 dist-test-slave-gkw7 krb5kdc[19057](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126333, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:12:13.417716 19386 init.cc:377] Logged in from keytab as kudu/127.18.154.2@KRBTEST.COM (short username kudu)
I20260807 18:12:13.419593 19386 webserver.cc:460] Webserver started at http://127.18.154.2:35507/ using document root <none> and password file <none>
I20260807 18:12:13.420342 19386 fs_manager.cc:359] Metadata directory not provided
I20260807 18:12:13.420498 19386 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:12:13.420886 19386 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:12:13.424155 19386 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126328808105-19048-0/minicluster-data/ts-1/data/instance:
uuid: "24e287c290e4409bacf3354e411c16ed"
format_stamp: "Formatted at 2026-08-07 18:12:13 on dist-test-slave-gkw7"
server_key: "6e321e27e36537c491a07e93fda88d44"
server_key_iv: "30d9c2aa79b7ec034d76bc428726cbde"
server_key_version: "encryptionkey@0"
I20260807 18:12:13.425123 19386 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126328808105-19048-0/minicluster-data/ts-1/wal/instance:
uuid: "24e287c290e4409bacf3354e411c16ed"
format_stamp: "Formatted at 2026-08-07 18:12:13 on dist-test-slave-gkw7"
server_key: "6e321e27e36537c491a07e93fda88d44"
server_key_iv: "30d9c2aa79b7ec034d76bc428726cbde"
server_key_version: "encryptionkey@0"
I20260807 18:12:13.430811 19386 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260807 18:12:13.435186 19406 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:12:13.436717 19386 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 18:12:13.436910 19386 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126328808105-19048-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126328808105-19048-0/minicluster-data/ts-1/wal
uuid: "24e287c290e4409bacf3354e411c16ed"
format_stamp: "Formatted at 2026-08-07 18:12:13 on dist-test-slave-gkw7"
server_key: "6e321e27e36537c491a07e93fda88d44"
server_key_iv: "30d9c2aa79b7ec034d76bc428726cbde"
server_key_version: "encryptionkey@0"
I20260807 18:12:13.437129 19386 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126328808105-19048-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126328808105-19048-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126328808105-19048-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:12:13.463507 19386 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:12:13.468430 19386 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:12:13.468744 19386 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:12:13.469907 19386 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:12:13.471357 19386 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:12:13.471464 19386 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:12:13.471616 19386 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:12:13.471692 19386 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:12:13.520923 19386 rpc_server.cc:307] RPC server started. Bound to: 127.18.154.2:44045
I20260807 18:12:13.520985 19574 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.154.2:44045 every 8 connection(s)
I20260807 18:12:13.522559 19386 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126328808105-19048-0/minicluster-data/ts-1/data/info.pb
I20260807 18:12:13.525887 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 19386
I20260807 18:12:13.526070 19048 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126328808105-19048-0/minicluster-data/ts-1/wal/instance
I20260807 18:12:13.526762 19048 external_mini_cluster.cc:1428] Setting key 4418340dc94f1deebb8a54b9d782a76e
Aug 07 18:12:13 dist-test-slave-gkw7 krb5kdc[19057](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126333, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.2@KRBTEST.COM for kudu/127.18.154.62@KRBTEST.COM
I20260807 18:12:13.547925 19578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:13.525509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.2:59441 (local address 127.18.154.62:40979)
0807 18:12:13.526132 (+ 623us) server_negotiation.cc:206] Beginning negotiation
0807 18:12:13.526137 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:12:13.527519 (+ 1382us) server_negotiation.cc:407] Connection header received
0807 18:12:13.529866 (+ 2347us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:12:13.529875 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:12:13.529966 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:12:13.530099 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:12:13.531780 (+ 1681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:13.533555 (+ 1775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:13.535400 (+ 1845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:13.536334 (+ 934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:13.536537 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:13.540589 (+ 4052us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:12:13.540598 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:12:13.540604 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:12:13.540653 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:12:13.544382 (+ 3729us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:12:13.545379 (+ 997us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:12:13.545383 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:12:13.545385 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:12:13.545507 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:12:13.546132 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:12:13.546135 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:12:13.546137 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:12:13.546421 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:12:13.546547 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0807 18:12:13.547499 (+ 952us) server_negotiation.cc:299] Negotiation successful
0807 18:12:13.547673 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":511,"thread_start_us":308,"threads_started":1}
I20260807 18:12:13.549592 19579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:13.526060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:40979 (local address 127.18.154.2:59441)
0807 18:12:13.527288 (+ 1228us) negotiation.cc:107] Waiting for socket to connect
0807 18:12:13.527331 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 18:12:13.529521 (+ 2190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:12:13.530427 (+ 906us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:12:13.530435 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:12:13.530954 (+ 519us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:12:13.531557 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:12:13.531576 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:12:13.533768 (+ 2192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:12:13.533774 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:12:13.535233 (+ 1459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:12:13.535246 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:12:13.536595 (+ 1349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:12:13.536600 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:12:13.536796 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:13.537642 (+ 846us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:12:13.537664 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:12:13.540329 (+ 2665us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:12:13.544542 (+ 4213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:12:13.544555 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:12:13.544576 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:12:13.545255 (+ 679us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:12:13.545657 (+ 402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:12:13.545665 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:12:13.545671 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:12:13.546003 (+ 332us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:12:13.546567 (+ 564us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:12:13.546579 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:12:13.547084 (+ 505us) client_negotiation.cc:769] Sending connection context
0807 18:12:13.547423 (+ 339us) client_negotiation.cc:240] Negotiation successful
0807 18:12:13.548090 (+ 667us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":893,"thread_start_us":637,"threads_started":1}
I20260807 18:12:13.551303 19575 heartbeater.cc:347] Connected to a master server at 127.18.154.62:40979
I20260807 18:12:13.551772 19575 heartbeater.cc:464] Registering TS with master...
I20260807 18:12:13.553085 19575 heartbeater.cc:511] Master 127.18.154.62:40979 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.18.154.3@KRBTEST.COM; defaulting to no policy
I20260807 18:12:13.554930 19120 ts_manager.cc:194] Registered new tserver with Master: 24e287c290e4409bacf3354e411c16ed (127.18.154.2:44045)
I20260807 18:12:13.555768 19120 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.154.2@KRBTEST.COM'} at 127.18.154.2:59441
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.154.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.154.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.3.keytab.
Entry for principal kudu/127.18.154.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.3.keytab.
I20260807 18:12:13.582890 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126328808105-19048-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126328808105-19048-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126328808105-19048-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126328808105-19048-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.154.3:0
--local_ip_for_outbound_sockets=127.18.154.3
--webserver_interface=127.18.154.3
--webserver_port=0
--tserver_master_addrs=127.18.154.62:40979
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.3.keytab
--principal=kudu/127.18.154.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.154.20:40709
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126328808105-19048-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126328808105-19048-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126328808105-19048-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:12:13.948253 19582 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:12:13.948540 19582 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:12:13.948642 19582 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:12:13.956849 19582 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:12:13.956967 19582 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:12:13.957088 19582 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.154.3
I20260807 18:12:13.967293 19582 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.154.20:40709
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126328808105-19048-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126328808105-19048-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.3.keytab
--principal=kudu/127.18.154.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.154.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126328808105-19048-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.154.3
--webserver_port=0
--tserver_master_addrs=127.18.154.62:40979
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.154.3
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126328808105-19048-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:12:13.968768 19582 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:12:13.970134 19582 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 18:12:13.975049 19376 heartbeater.cc:503] Master 127.18.154.62:40979 was elected leader, sending a full tablet report...
W20260807 18:12:13.984180 19593 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:12:13.984195 19590 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:12:13.984545 19591 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:12:13.985517 19582 server_base.cc:1034] running on GCE node
I20260807 18:12:13.986250 19582 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:12:13.987226 19582 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:12:13.988472 19582 hybrid_clock.cc:648] HybridClock initialized: now 1786126333988421 us; error 43 us; skew 500 ppm
I20260807 18:12:13.988834 19582 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:12:13 dist-test-slave-gkw7 krb5kdc[19057](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126333, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:12:13.992291 19582 init.cc:377] Logged in from keytab as kudu/127.18.154.3@KRBTEST.COM (short username kudu)
I20260807 18:12:13.994019 19582 webserver.cc:460] Webserver started at http://127.18.154.3:39503/ using document root <none> and password file <none>
I20260807 18:12:13.994741 19582 fs_manager.cc:359] Metadata directory not provided
I20260807 18:12:13.994848 19582 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:12:13.995134 19582 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:12:13.998219 19582 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126328808105-19048-0/minicluster-data/ts-2/data/instance:
uuid: "6c07d435b8fe42df98bec2ddd6a75706"
format_stamp: "Formatted at 2026-08-07 18:12:13 on dist-test-slave-gkw7"
server_key: "43e3071f4e541bab86d2d220a2d56ce8"
server_key_iv: "b8365cab6f7924c8aae7e4f2fa52f5a6"
server_key_version: "encryptionkey@0"
I20260807 18:12:13.998977 19582 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126328808105-19048-0/minicluster-data/ts-2/wal/instance:
uuid: "6c07d435b8fe42df98bec2ddd6a75706"
format_stamp: "Formatted at 2026-08-07 18:12:13 on dist-test-slave-gkw7"
server_key: "43e3071f4e541bab86d2d220a2d56ce8"
server_key_iv: "b8365cab6f7924c8aae7e4f2fa52f5a6"
server_key_version: "encryptionkey@0"
I20260807 18:12:14.004755 19582 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 18:12:14.008464 19609 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:12:14.009964 19582 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:12:14.010177 19582 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126328808105-19048-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126328808105-19048-0/minicluster-data/ts-2/wal
uuid: "6c07d435b8fe42df98bec2ddd6a75706"
format_stamp: "Formatted at 2026-08-07 18:12:13 on dist-test-slave-gkw7"
server_key: "43e3071f4e541bab86d2d220a2d56ce8"
server_key_iv: "b8365cab6f7924c8aae7e4f2fa52f5a6"
server_key_version: "encryptionkey@0"
I20260807 18:12:14.010387 19582 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126328808105-19048-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126328808105-19048-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126328808105-19048-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:12:14.023084 19582 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:12:14.027882 19582 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:12:14.028213 19582 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:12:14.029239 19582 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:12:14.030778 19582 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:12:14.030853 19582 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:12:14.031008 19582 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:12:14.031061 19582 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:12:14.073071 19582 rpc_server.cc:307] RPC server started. Bound to: 127.18.154.3:46515
I20260807 18:12:14.073120 19769 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.154.3:46515 every 8 connection(s)
I20260807 18:12:14.074582 19582 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126328808105-19048-0/minicluster-data/ts-2/data/info.pb
I20260807 18:12:14.084672 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 19582
I20260807 18:12:14.084872 19048 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786126328808105-19048-0/minicluster-data/ts-2/wal/instance
I20260807 18:12:14.085470 19048 external_mini_cluster.cc:1428] Setting key 69c92d35647e3181acf8f80a88ff46c2
Aug 07 18:12:14 dist-test-slave-gkw7 krb5kdc[19057](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126333, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.3@KRBTEST.COM for kudu/127.18.154.62@KRBTEST.COM
I20260807 18:12:14.096904 19773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:14.077590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.3:46901 (local address 127.18.154.62:40979)
0807 18:12:14.078040 (+ 450us) server_negotiation.cc:206] Beginning negotiation
0807 18:12:14.078045 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:12:14.079155 (+ 1110us) server_negotiation.cc:407] Connection header received
0807 18:12:14.081034 (+ 1879us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:12:14.081038 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:12:14.081117 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:12:14.081229 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:12:14.082827 (+ 1598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:14.083690 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:14.085954 (+ 2264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:14.086401 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:14.086516 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:14.091133 (+ 4617us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:12:14.091138 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:12:14.091141 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:12:14.091174 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:12:14.094180 (+ 3006us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:12:14.094836 (+ 656us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:12:14.094839 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:12:14.094842 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:12:14.094979 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:12:14.095382 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:12:14.095386 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:12:14.095387 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:12:14.095649 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:12:14.095758 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0807 18:12:14.096454 (+ 696us) server_negotiation.cc:299] Negotiation successful
0807 18:12:14.096626 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":345,"thread_start_us":236,"threads_started":1}
I20260807 18:12:14.097908 19774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:14.078214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:40979 (local address 127.18.154.3:46901)
0807 18:12:14.078983 (+ 769us) negotiation.cc:107] Waiting for socket to connect
0807 18:12:14.079030 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 18:12:14.080731 (+ 1701us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:12:14.081451 (+ 720us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:12:14.081459 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:12:14.082053 (+ 594us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:12:14.082659 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:12:14.082676 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:12:14.083819 (+ 1143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:12:14.083823 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:12:14.085779 (+ 1956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:12:14.085803 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:12:14.086668 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:12:14.086684 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:12:14.086895 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:14.087756 (+ 861us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:12:14.087789 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:12:14.090864 (+ 3075us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:12:14.094296 (+ 3432us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:12:14.094302 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:12:14.094314 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:12:14.094721 (+ 407us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:12:14.095075 (+ 354us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:12:14.095078 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:12:14.095080 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:12:14.095280 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:12:14.095756 (+ 476us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:12:14.095762 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:12:14.096060 (+ 298us) client_negotiation.cc:769] Sending connection context
0807 18:12:14.096301 (+ 241us) client_negotiation.cc:240] Negotiation successful
0807 18:12:14.096684 (+ 383us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":541,"thread_start_us":339,"threads_started":1}
I20260807 18:12:14.099478 19770 heartbeater.cc:347] Connected to a master server at 127.18.154.62:40979
I20260807 18:12:14.099812 19770 heartbeater.cc:464] Registering TS with master...
I20260807 18:12:14.100526 19770 heartbeater.cc:511] Master 127.18.154.62:40979 requested a full tablet report, sending...
I20260807 18:12:14.102185 19120 ts_manager.cc:194] Registered new tserver with Master: 6c07d435b8fe42df98bec2ddd6a75706 (127.18.154.3:46515)
I20260807 18:12:14.103333 19120 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.154.3@KRBTEST.COM'} at 127.18.154.3:46901
I20260807 18:12:14.103374 19048 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 18:12:14 dist-test-slave-gkw7 krb5kdc[19057](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126334, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 18:12:14 dist-test-slave-gkw7 krb5kdc[19057](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126334, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.18.154.1@KRBTEST.COM
I20260807 18:12:14.169425 19781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:14.146546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55356 (local address 127.18.154.1:41259)
0807 18:12:14.147276 (+ 730us) server_negotiation.cc:206] Beginning negotiation
0807 18:12:14.147283 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:12:14.147312 (+ 29us) server_negotiation.cc:407] Connection header received
0807 18:12:14.147377 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:12:14.147384 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:12:14.147788 (+ 404us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:12:14.147976 (+ 188us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:12:14.149500 (+ 1524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:14.151456 (+ 1956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:14.153963 (+ 2507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:14.154904 (+ 941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:14.155030 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:14.160722 (+ 5692us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:12:14.160739 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:12:14.160763 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:12:14.160818 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:12:14.165372 (+ 4554us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:12:14.166362 (+ 990us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:12:14.166373 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:12:14.166380 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:12:14.166574 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:12:14.167083 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:12:14.167089 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:12:14.167094 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:12:14.167637 (+ 543us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:12:14.168101 (+ 464us) server_negotiation.cc:1035] Waiting for connection context
0807 18:12:14.168320 (+ 219us) server_negotiation.cc:299] Negotiation successful
0807 18:12:14.168846 (+ 526us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":553,"thread_start_us":389,"threads_started":1}
W20260807 18:12:14.170989 19263 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55356
Aug 07 18:12:14 dist-test-slave-gkw7 krb5kdc[19057](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126334, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 18:12:14 dist-test-slave-gkw7 krb5kdc[19057](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126334, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.154.1@KRBTEST.COM
I20260807 18:12:14.227504 19781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:14.212961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55362 (local address 127.18.154.1:41259)
0807 18:12:14.213221 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0807 18:12:14.213229 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:12:14.213467 (+ 238us) server_negotiation.cc:407] Connection header received
0807 18:12:14.213773 (+ 306us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:12:14.213778 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:12:14.213878 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:12:14.213998 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:12:14.215565 (+ 1567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:14.216297 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:14.217810 (+ 1513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:14.218304 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:14.218404 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:14.222657 (+ 4253us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:12:14.222663 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:12:14.222667 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:12:14.222704 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:12:14.225152 (+ 2448us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:12:14.225890 (+ 738us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:12:14.225895 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:12:14.225897 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:12:14.226039 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:12:14.226339 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:12:14.226342 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:12:14.226344 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:12:14.226610 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:12:14.226793 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0807 18:12:14.227022 (+ 229us) server_negotiation.cc:299] Negotiation successful
0807 18:12:14.227196 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100}
I20260807 18:12:14.229835 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 19207
I20260807 18:12:14.243924 19781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:14.231087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55370 (local address 127.18.154.1:41259)
0807 18:12:14.231341 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0807 18:12:14.231348 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:12:14.231623 (+ 275us) server_negotiation.cc:407] Connection header received
0807 18:12:14.231804 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:12:14.231813 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:12:14.231871 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:12:14.231974 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:12:14.233271 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:14.234066 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:14.235376 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:14.235801 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:14.235942 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:14.238592 (+ 2650us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:12:14.238597 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:12:14.238600 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:12:14.238634 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:12:14.241085 (+ 2451us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:12:14.241905 (+ 820us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:12:14.241913 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:12:14.241916 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:12:14.242072 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:12:14.242455 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:12:14.242460 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:12:14.242463 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:12:14.242736 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:12:14.242991 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0807 18:12:14.243297 (+ 306us) server_negotiation.cc:299] Negotiation successful
0807 18:12:14.243511 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260807 18:12:14.244824 19371 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:12:14.337255 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 19207
I20260807 18:12:14.363540 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 19386
Aug 07 18:12:14 dist-test-slave-gkw7 krb5kdc[19057](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126334, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.154.2@KRBTEST.COM
I20260807 18:12:14.380319 19791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:14.364538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50972 (local address 127.18.154.2:44045)
0807 18:12:14.365126 (+ 588us) server_negotiation.cc:206] Beginning negotiation
0807 18:12:14.365133 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:12:14.365160 (+ 27us) server_negotiation.cc:407] Connection header received
0807 18:12:14.365228 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:12:14.365234 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:12:14.365446 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:12:14.365639 (+ 193us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:12:14.366821 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:14.367913 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:14.369570 (+ 1657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:14.370133 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:14.370251 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:14.374423 (+ 4172us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:12:14.374435 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:12:14.374450 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:12:14.374492 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:12:14.377445 (+ 2953us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:12:14.378157 (+ 712us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:12:14.378165 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:12:14.378167 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:12:14.378322 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:12:14.378763 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:12:14.378767 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:12:14.378770 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:12:14.379180 (+ 410us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:12:14.379428 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0807 18:12:14.379808 (+ 380us) server_negotiation.cc:299] Negotiation successful
0807 18:12:14.380017 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":431,"thread_start_us":320,"threads_started":1}
I20260807 18:12:14.381742 19564 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:12:14.465767 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 19386
I20260807 18:12:14.491453 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 19582
Aug 07 18:12:14 dist-test-slave-gkw7 krb5kdc[19057](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126334, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.154.3@KRBTEST.COM
I20260807 18:12:14.508031 19797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:14.492570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60638 (local address 127.18.154.3:46515)
0807 18:12:14.493261 (+ 691us) server_negotiation.cc:206] Beginning negotiation
0807 18:12:14.493269 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:12:14.493299 (+ 30us) server_negotiation.cc:407] Connection header received
0807 18:12:14.493359 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:12:14.493365 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:12:14.493705 (+ 340us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:12:14.493871 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:12:14.495134 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:14.496217 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:14.497480 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:14.498102 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:14.498208 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:14.502433 (+ 4225us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:12:14.502447 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:12:14.502463 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:12:14.502498 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:12:14.505280 (+ 2782us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:12:14.506028 (+ 748us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:12:14.506036 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:12:14.506038 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:12:14.506167 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:12:14.506487 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:12:14.506490 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:12:14.506492 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:12:14.506854 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:12:14.507094 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0807 18:12:14.507397 (+ 303us) server_negotiation.cc:299] Negotiation successful
0807 18:12:14.507602 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":536,"thread_start_us":374,"threads_started":1}
I20260807 18:12:14.509625 19762 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:12:14.593174 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 19582
I20260807 18:12:14.614321 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 19073
I20260807 18:12:14.615648 19162 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:12:14.721500 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 19073
2026-08-07T18:12:14Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (5818 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126328808105-19048-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:12:14 dist-test-slave-gkw7 krb5kdc[19805](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:12:14 dist-test-slave-gkw7 krb5kdc[19805](info): set up 2 sockets
Aug 07 18:12:14 dist-test-slave-gkw7 krb5kdc[19805](info): commencing operation
krb5kdc: starting...
W20260807 18:12:16.043994 19048 mini_kdc.cc:121] Time spent starting KDC: real 1.274s user 0.002s sys 0.018s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126328808105-19048-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126328808105-19048-0/krb5kdc/test-user.keytab.
Aug 07 18:12:16 dist-test-slave-gkw7 krb5kdc[19805](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126336, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:12:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:12:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.154.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.154.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.154.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.62.keytab.
Entry for principal kudu/127.18.154.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.62.keytab.
I20260807 18:12:16.266559 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126328808105-19048-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126328808105-19048-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126328808105-19048-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.154.62:44877
--webserver_interface=127.18.154.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.62.keytab
--principal=kudu/127.18.154.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.154.20:42471
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.154.62:44877
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126328808105-19048-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126328808105-19048-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126328808105-19048-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:12:16.615386 19822 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:12:16.615741 19822 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:12:16.615906 19822 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:12:16.623369 19822 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:12:16.623508 19822 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:12:16.623561 19822 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:12:16.623621 19822 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:12:16.623665 19822 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:12:16.634132 19822 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.154.20:42471
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126328808105-19048-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126328808105-19048-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.154.62:44877
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.62.keytab
--principal=kudu/127.18.154.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.154.62:44877
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.154.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126328808105-19048-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:12:16.635658 19822 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:12:16.637215 19822 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:12:16.649884 19828 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:12:16.650151 19831 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:12:16.650300 19829 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:12:16.651587 19822 server_base.cc:1034] running on GCE node
I20260807 18:12:16.652408 19822 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:12:16.653978 19822 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:12:16.655186 19822 hybrid_clock.cc:648] HybridClock initialized: now 1786126336655139 us; error 38 us; skew 500 ppm
I20260807 18:12:16.655589 19822 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:12:16 dist-test-slave-gkw7 krb5kdc[19805](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126336, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:12:16.659005 19822 init.cc:377] Logged in from keytab as kudu/127.18.154.62@KRBTEST.COM (short username kudu)
I20260807 18:12:16.660692 19822 webserver.cc:460] Webserver started at http://127.18.154.62:40019/ using document root <none> and password file <none>
I20260807 18:12:16.661446 19822 fs_manager.cc:359] Metadata directory not provided
I20260807 18:12:16.661625 19822 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:12:16.661989 19822 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:12:16.665151 19822 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126328808105-19048-0/minicluster-data/master-0/data/instance:
uuid: "5498898fbf5e4357bbcafc9e488f2392"
format_stamp: "Formatted at 2026-08-07 18:12:16 on dist-test-slave-gkw7"
server_key: "6636b2837293d2a77057a365cba76454"
server_key_iv: "97de01934ec8ad7166ebed1189e7a9cd"
server_key_version: "encryptionkey@0"
I20260807 18:12:16.665959 19822 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126328808105-19048-0/minicluster-data/master-0/wal/instance:
uuid: "5498898fbf5e4357bbcafc9e488f2392"
format_stamp: "Formatted at 2026-08-07 18:12:16 on dist-test-slave-gkw7"
server_key: "6636b2837293d2a77057a365cba76454"
server_key_iv: "97de01934ec8ad7166ebed1189e7a9cd"
server_key_version: "encryptionkey@0"
I20260807 18:12:16.671677 19822 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 18:12:16.675457 19845 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:12:16.676855 19822 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 18:12:16.677054 19822 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126328808105-19048-0/minicluster-data/master-0/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126328808105-19048-0/minicluster-data/master-0/wal
uuid: "5498898fbf5e4357bbcafc9e488f2392"
format_stamp: "Formatted at 2026-08-07 18:12:16 on dist-test-slave-gkw7"
server_key: "6636b2837293d2a77057a365cba76454"
server_key_iv: "97de01934ec8ad7166ebed1189e7a9cd"
server_key_version: "encryptionkey@0"
I20260807 18:12:16.677296 19822 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126328808105-19048-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126328808105-19048-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126328808105-19048-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:12:16.710870 19822 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:12:16.715772 19822 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:12:16.716111 19822 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:12:16.737293 19822 rpc_server.cc:307] RPC server started. Bound to: 127.18.154.62:44877
I20260807 18:12:16.737287 19915 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.154.62:44877 every 8 connection(s)
I20260807 18:12:16.738900 19822 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
I20260807 18:12:16.743136 19916 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:12:16.748781 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 19822
I20260807 18:12:16.749029 19048 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126328808105-19048-0/minicluster-data/master-0/wal/instance
I20260807 18:12:16.749779 19048 external_mini_cluster.cc:1428] Setting key 4c1c98a958b9f88d5a7d894fe18d4e7e
I20260807 18:12:16.756379 19916 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5498898fbf5e4357bbcafc9e488f2392: Bootstrap starting.
I20260807 18:12:16.760771 19916 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5498898fbf5e4357bbcafc9e488f2392: Neither blocks nor log segments found. Creating new log.
I20260807 18:12:16.762006 19916 log.cc:824] T 00000000000000000000000000000000 P 5498898fbf5e4357bbcafc9e488f2392: Log is configured to *not* fsync() on all Append() calls
I20260807 18:12:16.765157 19916 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5498898fbf5e4357bbcafc9e488f2392: No bootstrap required, opened a new log
Aug 07 18:12:16 dist-test-slave-gkw7 krb5kdc[19805](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126336, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.154.62@KRBTEST.COM
I20260807 18:12:16.774935 19916 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5498898fbf5e4357bbcafc9e488f2392 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5498898fbf5e4357bbcafc9e488f2392" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 44877 } }
I20260807 18:12:16.775416 19916 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5498898fbf5e4357bbcafc9e488f2392 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:12:16.775552 19916 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5498898fbf5e4357bbcafc9e488f2392 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5498898fbf5e4357bbcafc9e488f2392, State: Initialized, Role: FOLLOWER
I20260807 18:12:16.776175 19916 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5498898fbf5e4357bbcafc9e488f2392 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5498898fbf5e4357bbcafc9e488f2392" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 44877 } }
I20260807 18:12:16.776432 19916 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5498898fbf5e4357bbcafc9e488f2392 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:12:16.776592 19916 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5498898fbf5e4357bbcafc9e488f2392 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:12:16.776635 19926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:16.752077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40218 (local address 127.18.154.62:44877)
0807 18:12:16.753095 (+ 1018us) server_negotiation.cc:206] Beginning negotiation
0807 18:12:16.753112 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 18:12:16.753176 (+ 64us) server_negotiation.cc:407] Connection header received
0807 18:12:16.755727 (+ 2551us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:12:16.755758 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:12:16.756330 (+ 572us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:12:16.756855 (+ 525us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:12:16.758375 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:16.760436 (+ 2061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:16.762032 (+ 1596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:16.763043 (+ 1011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:16.763193 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:16.767998 (+ 4805us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:12:16.768006 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:12:16.768023 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:12:16.768064 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:12:16.771678 (+ 3614us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:12:16.772581 (+ 903us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:12:16.772589 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:12:16.772600 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:12:16.772811 (+ 211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:12:16.773260 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:12:16.773266 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:12:16.773268 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:12:16.773955 (+ 687us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:12:16.774236 (+ 281us) server_negotiation.cc:1035] Waiting for connection context
0807 18:12:16.774766 (+ 530us) server_negotiation.cc:299] Negotiation successful
0807 18:12:16.775206 (+ 440us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":726,"thread_start_us":459,"threads_started":1}
I20260807 18:12:16.776804 19916 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5498898fbf5e4357bbcafc9e488f2392 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:12:16.779300 19916 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5498898fbf5e4357bbcafc9e488f2392 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5498898fbf5e4357bbcafc9e488f2392" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 44877 } }
I20260807 18:12:16.779906 19916 leader_election.cc:302] T 00000000000000000000000000000000 P 5498898fbf5e4357bbcafc9e488f2392 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5498898fbf5e4357bbcafc9e488f2392; no voters:
I20260807 18:12:16.780526 19916 leader_election.cc:288] T 00000000000000000000000000000000 P 5498898fbf5e4357bbcafc9e488f2392 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:12:16.780676 19928 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5498898fbf5e4357bbcafc9e488f2392 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:12:16.781489 19928 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5498898fbf5e4357bbcafc9e488f2392 [term 1 LEADER]: Becoming Leader. State: Replica: 5498898fbf5e4357bbcafc9e488f2392, State: Running, Role: LEADER
I20260807 18:12:16.782189 19928 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5498898fbf5e4357bbcafc9e488f2392 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5498898fbf5e4357bbcafc9e488f2392" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 44877 } }
I20260807 18:12:16.782960 19916 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5498898fbf5e4357bbcafc9e488f2392 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:12:16.785248 19929 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5498898fbf5e4357bbcafc9e488f2392 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5498898fbf5e4357bbcafc9e488f2392" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5498898fbf5e4357bbcafc9e488f2392" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 44877 } } }
I20260807 18:12:16.785161 19930 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5498898fbf5e4357bbcafc9e488f2392 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5498898fbf5e4357bbcafc9e488f2392. Latest consensus state: current_term: 1 leader_uuid: "5498898fbf5e4357bbcafc9e488f2392" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5498898fbf5e4357bbcafc9e488f2392" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 44877 } } }
I20260807 18:12:16.785722 19929 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5498898fbf5e4357bbcafc9e488f2392 [sys.catalog]: This master's current role is: LEADER
I20260807 18:12:16.785735 19930 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5498898fbf5e4357bbcafc9e488f2392 [sys.catalog]: This master's current role is: LEADER
I20260807 18:12:16.786851 19936 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:12:16.792946 19936 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:12:16.802778 19936 catalog_manager.cc:1347] Generated new cluster ID: 85188457bc0c4cdabd2be62334254794
I20260807 18:12:16.802918 19936 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:12:16.833344 19936 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:12:16.835357 19936 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:12:16.850126 19936 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5498898fbf5e4357bbcafc9e488f2392: Generated new TSK 0
I20260807 18:12:16.851289 19936 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.154.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.154.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.154.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.1.keytab.
Entry for principal kudu/127.18.154.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.1.keytab.
I20260807 18:12:16.906481 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126328808105-19048-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126328808105-19048-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126328808105-19048-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126328808105-19048-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.154.1:0
--local_ip_for_outbound_sockets=127.18.154.1
--webserver_interface=127.18.154.1
--webserver_port=0
--tserver_master_addrs=127.18.154.62:44877
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.1.keytab
--principal=kudu/127.18.154.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.154.20:42471
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126328808105-19048-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126328808105-19048-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126328808105-19048-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:12:17.262590 19956 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:12:17.262953 19956 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:12:17.263137 19956 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:12:17.270646 19956 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:12:17.270823 19956 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:12:17.271020 19956 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.154.1
I20260807 18:12:17.283804 19956 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.154.20:42471
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126328808105-19048-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126328808105-19048-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.1.keytab
--principal=kudu/127.18.154.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.154.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126328808105-19048-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.154.1
--webserver_port=0
--tserver_master_addrs=127.18.154.62:44877
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.154.1
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126328808105-19048-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:12:17.285281 19956 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:12:17.286765 19956 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:12:17.300884 19962 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:12:17.300839 19964 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:12:17.300804 19967 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:12:17.301658 19956 server_base.cc:1034] running on GCE node
I20260807 18:12:17.302327 19956 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:12:17.303364 19956 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:12:17.304610 19956 hybrid_clock.cc:648] HybridClock initialized: now 1786126337304565 us; error 59 us; skew 500 ppm
I20260807 18:12:17.304960 19956 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:12:17 dist-test-slave-gkw7 krb5kdc[19805](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126337, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:12:17.313514 19956 init.cc:377] Logged in from keytab as kudu/127.18.154.1@KRBTEST.COM (short username kudu)
I20260807 18:12:17.315271 19956 webserver.cc:460] Webserver started at http://127.18.154.1:39211/ using document root <none> and password file <none>
I20260807 18:12:17.315992 19956 fs_manager.cc:359] Metadata directory not provided
I20260807 18:12:17.316103 19956 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:12:17.316406 19956 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:12:17.319633 19956 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126328808105-19048-0/minicluster-data/ts-0/data/instance:
uuid: "03217377ae944cd39f87ad5119c21e9d"
format_stamp: "Formatted at 2026-08-07 18:12:17 on dist-test-slave-gkw7"
server_key: "f71d35491bf7c5b2d50a19d9d2790f06"
server_key_iv: "d8ce473d4636f8367a448fd188517075"
server_key_version: "encryptionkey@0"
I20260807 18:12:17.320442 19956 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126328808105-19048-0/minicluster-data/ts-0/wal/instance:
uuid: "03217377ae944cd39f87ad5119c21e9d"
format_stamp: "Formatted at 2026-08-07 18:12:17 on dist-test-slave-gkw7"
server_key: "f71d35491bf7c5b2d50a19d9d2790f06"
server_key_iv: "d8ce473d4636f8367a448fd188517075"
server_key_version: "encryptionkey@0"
I20260807 18:12:17.326018 19956 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.007s
I20260807 18:12:17.329883 19975 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:12:17.331390 19956 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 18:12:17.331570 19956 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126328808105-19048-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126328808105-19048-0/minicluster-data/ts-0/wal
uuid: "03217377ae944cd39f87ad5119c21e9d"
format_stamp: "Formatted at 2026-08-07 18:12:17 on dist-test-slave-gkw7"
server_key: "f71d35491bf7c5b2d50a19d9d2790f06"
server_key_iv: "d8ce473d4636f8367a448fd188517075"
server_key_version: "encryptionkey@0"
I20260807 18:12:17.331795 19956 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126328808105-19048-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126328808105-19048-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126328808105-19048-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:12:17.369910 19956 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:12:17.374569 19956 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:12:17.374958 19956 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:12:17.376140 19956 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:12:17.377600 19956 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:12:17.377684 19956 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:12:17.377858 19956 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:12:17.377923 19956 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:12:17.413872 19956 rpc_server.cc:307] RPC server started. Bound to: 127.18.154.1:40819
I20260807 18:12:17.413921 20141 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.154.1:40819 every 8 connection(s)
I20260807 18:12:17.415366 19956 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126328808105-19048-0/minicluster-data/ts-0/data/info.pb
I20260807 18:12:17.419297 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 19956
I20260807 18:12:17.419503 19048 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126328808105-19048-0/minicluster-data/ts-0/wal/instance
I20260807 18:12:17.420352 19048 external_mini_cluster.cc:1428] Setting key dd371f6331ddef98ff2033f3f853252c
Aug 07 18:12:17 dist-test-slave-gkw7 krb5kdc[19805](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126337, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.1@KRBTEST.COM for kudu/127.18.154.62@KRBTEST.COM
I20260807 18:12:17.437350 20146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:17.418083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.1:36309 (local address 127.18.154.62:44877)
0807 18:12:17.418586 (+ 503us) server_negotiation.cc:206] Beginning negotiation
0807 18:12:17.418591 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:12:17.421065 (+ 2474us) server_negotiation.cc:407] Connection header received
0807 18:12:17.422130 (+ 1065us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:12:17.422136 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:12:17.422232 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:12:17.422440 (+ 208us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:12:17.424002 (+ 1562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:17.424750 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:17.426468 (+ 1718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:17.426888 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:17.427035 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:17.431023 (+ 3988us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:12:17.431029 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:12:17.431033 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:12:17.431067 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:12:17.433869 (+ 2802us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:12:17.434687 (+ 818us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:12:17.434691 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:12:17.434694 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:12:17.434822 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:12:17.435314 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:12:17.435318 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:12:17.435320 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:12:17.435668 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:12:17.435850 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0807 18:12:17.436703 (+ 853us) server_negotiation.cc:299] Negotiation successful
0807 18:12:17.436954 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":403,"thread_start_us":285,"threads_started":1}
I20260807 18:12:17.438457 20148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:17.418542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:44877 (local address 127.18.154.1:36309)
0807 18:12:17.419415 (+ 873us) negotiation.cc:107] Waiting for socket to connect
0807 18:12:17.419482 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0807 18:12:17.421859 (+ 2377us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:12:17.422689 (+ 830us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:12:17.422697 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:12:17.423233 (+ 536us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:12:17.423819 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:12:17.423842 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:12:17.424924 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:12:17.424928 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:12:17.426306 (+ 1378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:12:17.426319 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:12:17.427130 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:12:17.427138 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:12:17.427393 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:17.428136 (+ 743us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:12:17.428161 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:12:17.430767 (+ 2606us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:12:17.434027 (+ 3260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:12:17.434038 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:12:17.434059 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:12:17.434539 (+ 480us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:12:17.434943 (+ 404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:12:17.434950 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:12:17.434955 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:12:17.435175 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:12:17.435820 (+ 645us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:12:17.435829 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:12:17.436290 (+ 461us) client_negotiation.cc:769] Sending connection context
0807 18:12:17.436576 (+ 286us) client_negotiation.cc:240] Negotiation successful
0807 18:12:17.437222 (+ 646us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":546,"thread_start_us":314,"threads_started":1}
WARNING: no policy specified for kudu/127.18.154.2@KRBTEST.COM; defaulting to no policy
I20260807 18:12:17.440300 20142 heartbeater.cc:347] Connected to a master server at 127.18.154.62:44877
I20260807 18:12:17.440591 20142 heartbeater.cc:464] Registering TS with master...
I20260807 18:12:17.441390 20142 heartbeater.cc:511] Master 127.18.154.62:44877 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.154.2@KRBTEST.COM" created.
I20260807 18:12:17.444065 19867 ts_manager.cc:194] Registered new tserver with Master: 03217377ae944cd39f87ad5119c21e9d (127.18.154.1:40819)
I20260807 18:12:17.445824 19867 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.154.1@KRBTEST.COM'} at 127.18.154.1:36309
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.154.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.2.keytab.
Entry for principal kudu/127.18.154.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.2.keytab.
I20260807 18:12:17.468622 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126328808105-19048-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126328808105-19048-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126328808105-19048-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126328808105-19048-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.154.2:0
--local_ip_for_outbound_sockets=127.18.154.2
--webserver_interface=127.18.154.2
--webserver_port=0
--tserver_master_addrs=127.18.154.62:44877
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.2.keytab
--principal=kudu/127.18.154.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.154.20:42471
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126328808105-19048-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126328808105-19048-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126328808105-19048-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:12:17.814092 20151 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:12:17.814386 20151 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:12:17.814556 20151 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:12:17.821933 20151 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:12:17.822050 20151 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:12:17.822233 20151 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.154.2
I20260807 18:12:17.832192 20151 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.154.20:42471
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126328808105-19048-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126328808105-19048-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.2.keytab
--principal=kudu/127.18.154.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.154.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126328808105-19048-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.154.2
--webserver_port=0
--tserver_master_addrs=127.18.154.62:44877
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.154.2
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126328808105-19048-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:12:17.833626 20151 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:12:17.834987 20151 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:12:17.848614 20159 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:12:17.848567 20160 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:12:17.849126 20162 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:12:17.850111 20151 server_base.cc:1034] running on GCE node
I20260807 18:12:17.850757 20151 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:12:17.851846 20151 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:12:17.853083 20151 hybrid_clock.cc:648] HybridClock initialized: now 1786126337852989 us; error 90 us; skew 500 ppm
I20260807 18:12:17.853423 20151 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:12:17 dist-test-slave-gkw7 krb5kdc[19805](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126337, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:12:17.856834 20151 init.cc:377] Logged in from keytab as kudu/127.18.154.2@KRBTEST.COM (short username kudu)
I20260807 18:12:17.858639 20151 webserver.cc:460] Webserver started at http://127.18.154.2:38153/ using document root <none> and password file <none>
I20260807 18:12:17.859404 20151 fs_manager.cc:359] Metadata directory not provided
I20260807 18:12:17.859555 20151 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:12:17.859901 20151 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:12:17.863062 20151 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126328808105-19048-0/minicluster-data/ts-1/data/instance:
uuid: "6f7ed67d931d4357a07d5c38f36d91a3"
format_stamp: "Formatted at 2026-08-07 18:12:17 on dist-test-slave-gkw7"
server_key: "00e1aa24260e68ff24e5b7f82e6ab592"
server_key_iv: "39774a96fcd7bdb47fd1beaeae21f191"
server_key_version: "encryptionkey@0"
I20260807 18:12:17.863893 20151 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126328808105-19048-0/minicluster-data/ts-1/wal/instance:
uuid: "6f7ed67d931d4357a07d5c38f36d91a3"
format_stamp: "Formatted at 2026-08-07 18:12:17 on dist-test-slave-gkw7"
server_key: "00e1aa24260e68ff24e5b7f82e6ab592"
server_key_iv: "39774a96fcd7bdb47fd1beaeae21f191"
server_key_version: "encryptionkey@0"
I20260807 18:12:17.869345 20151 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260807 18:12:17.873111 20174 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:12:17.874577 20151 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:12:17.874788 20151 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126328808105-19048-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126328808105-19048-0/minicluster-data/ts-1/wal
uuid: "6f7ed67d931d4357a07d5c38f36d91a3"
format_stamp: "Formatted at 2026-08-07 18:12:17 on dist-test-slave-gkw7"
server_key: "00e1aa24260e68ff24e5b7f82e6ab592"
server_key_iv: "39774a96fcd7bdb47fd1beaeae21f191"
server_key_version: "encryptionkey@0"
I20260807 18:12:17.875007 20151 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126328808105-19048-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126328808105-19048-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126328808105-19048-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:12:17.888316 20151 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:12:17.893016 20151 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:12:17.893348 20151 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:12:17.894452 20151 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:12:17.896039 20151 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:12:17.896148 20151 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:12:17.896304 20151 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:12:17.896380 20151 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:12:17.935150 20151 rpc_server.cc:307] RPC server started. Bound to: 127.18.154.2:39287
I20260807 18:12:17.935190 20332 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.154.2:39287 every 8 connection(s)
I20260807 18:12:17.936740 20151 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126328808105-19048-0/minicluster-data/ts-1/data/info.pb
I20260807 18:12:17.941502 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 20151
I20260807 18:12:17.941725 19048 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126328808105-19048-0/minicluster-data/ts-1/wal/instance
I20260807 18:12:17.942272 19048 external_mini_cluster.cc:1428] Setting key 2acb800e0c2442d50ecf9dd204409fb8
Aug 07 18:12:17 dist-test-slave-gkw7 krb5kdc[19805](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126337, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.2@KRBTEST.COM for kudu/127.18.154.62@KRBTEST.COM
I20260807 18:12:17.959993 20339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:17.939954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.2:34749 (local address 127.18.154.62:44877)
0807 18:12:17.940472 (+ 518us) server_negotiation.cc:206] Beginning negotiation
0807 18:12:17.940477 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:12:17.944174 (+ 3697us) server_negotiation.cc:407] Connection header received
0807 18:12:17.944230 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:12:17.944234 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:12:17.944303 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:12:17.944430 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:12:17.946266 (+ 1836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:17.947092 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:17.948865 (+ 1773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:17.949371 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:17.949528 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:17.953800 (+ 4272us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:12:17.953805 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:12:17.953809 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:12:17.953845 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:12:17.956537 (+ 2692us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:12:17.957375 (+ 838us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:12:17.957379 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:12:17.957382 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:12:17.957517 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:12:17.958057 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:12:17.958063 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:12:17.958066 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:12:17.958360 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:12:17.958534 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0807 18:12:17.959346 (+ 812us) server_negotiation.cc:299] Negotiation successful
0807 18:12:17.959611 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":403,"thread_start_us":278,"threads_started":1}
I20260807 18:12:17.961002 20340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:17.940654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:44877 (local address 127.18.154.2:34749)
0807 18:12:17.941449 (+ 795us) negotiation.cc:107] Waiting for socket to connect
0807 18:12:17.941492 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 18:12:17.943778 (+ 2286us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:12:17.944718 (+ 940us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:12:17.944727 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:12:17.945324 (+ 597us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:12:17.946070 (+ 746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:12:17.946089 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:12:17.947271 (+ 1182us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:12:17.947276 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:12:17.948677 (+ 1401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:12:17.948691 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:12:17.949666 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:12:17.949677 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:12:17.949960 (+ 283us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:17.950754 (+ 794us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:12:17.950779 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:12:17.953463 (+ 2684us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:12:17.956710 (+ 3247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:12:17.956729 (+ 19us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:12:17.956756 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:12:17.957226 (+ 470us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:12:17.957692 (+ 466us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:12:17.957697 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:12:17.957701 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:12:17.957924 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:12:17.958497 (+ 573us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:12:17.958506 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:12:17.958937 (+ 431us) client_negotiation.cc:769] Sending connection context
0807 18:12:17.959222 (+ 285us) client_negotiation.cc:240] Negotiation successful
0807 18:12:17.959839 (+ 617us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":538,"thread_start_us":318,"threads_started":1}
WARNING: no policy specified for kudu/127.18.154.3@KRBTEST.COM; defaulting to no policy
I20260807 18:12:17.962610 20335 heartbeater.cc:347] Connected to a master server at 127.18.154.62:44877
I20260807 18:12:17.962968 20335 heartbeater.cc:464] Registering TS with master...
I20260807 18:12:17.963743 20335 heartbeater.cc:511] Master 127.18.154.62:44877 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.154.3@KRBTEST.COM" created.
I20260807 18:12:17.965338 19867 ts_manager.cc:194] Registered new tserver with Master: 6f7ed67d931d4357a07d5c38f36d91a3 (127.18.154.2:39287)
I20260807 18:12:17.966215 19867 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.154.2@KRBTEST.COM'} at 127.18.154.2:34749
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.154.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.3.keytab.
Entry for principal kudu/127.18.154.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.3.keytab.
I20260807 18:12:17.990753 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126328808105-19048-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126328808105-19048-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126328808105-19048-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126328808105-19048-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.154.3:0
--local_ip_for_outbound_sockets=127.18.154.3
--webserver_interface=127.18.154.3
--webserver_port=0
--tserver_master_addrs=127.18.154.62:44877
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.3.keytab
--principal=kudu/127.18.154.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.154.20:42471
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126328808105-19048-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126328808105-19048-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126328808105-19048-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:12:18.334477 20347 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:12:18.334780 20347 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:12:18.334996 20347 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:12:18.342581 20347 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:12:18.342696 20347 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:12:18.342919 20347 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.154.3
I20260807 18:12:18.352897 20347 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.154.20:42471
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126328808105-19048-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126328808105-19048-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.3.keytab
--principal=kudu/127.18.154.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.154.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126328808105-19048-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.154.3
--webserver_port=0
--tserver_master_addrs=127.18.154.62:44877
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.154.3
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126328808105-19048-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:12:18.354295 20347 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:12:18.355677 20347 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:12:18.369731 20357 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:12:18.369803 20353 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:12:18.369685 20354 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:12:18.370615 20347 server_base.cc:1034] running on GCE node
I20260807 18:12:18.371271 20347 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:12:18.372210 20347 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:12:18.373413 20347 hybrid_clock.cc:648] HybridClock initialized: now 1786126338373367 us; error 40 us; skew 500 ppm
I20260807 18:12:18.373754 20347 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:12:18 dist-test-slave-gkw7 krb5kdc[19805](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126338, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:12:18.377032 20347 init.cc:377] Logged in from keytab as kudu/127.18.154.3@KRBTEST.COM (short username kudu)
I20260807 18:12:18.378712 20347 webserver.cc:460] Webserver started at http://127.18.154.3:46525/ using document root <none> and password file <none>
I20260807 18:12:18.379380 20347 fs_manager.cc:359] Metadata directory not provided
I20260807 18:12:18.379478 20347 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:12:18.379824 20347 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:12:18.383082 20347 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126328808105-19048-0/minicluster-data/ts-2/data/instance:
uuid: "443bda07921549e1b17da4a04c0eadb5"
format_stamp: "Formatted at 2026-08-07 18:12:18 on dist-test-slave-gkw7"
server_key: "b74d63fa20594aae466b4bc3c167ae82"
server_key_iv: "92e8fa3dd1a13512f3b24b62b3b18387"
server_key_version: "encryptionkey@0"
I20260807 18:12:18.383961 20347 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126328808105-19048-0/minicluster-data/ts-2/wal/instance:
uuid: "443bda07921549e1b17da4a04c0eadb5"
format_stamp: "Formatted at 2026-08-07 18:12:18 on dist-test-slave-gkw7"
server_key: "b74d63fa20594aae466b4bc3c167ae82"
server_key_iv: "92e8fa3dd1a13512f3b24b62b3b18387"
server_key_version: "encryptionkey@0"
I20260807 18:12:18.389580 20347 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260807 18:12:18.393291 20367 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:12:18.394774 20347 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260807 18:12:18.394954 20347 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126328808105-19048-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126328808105-19048-0/minicluster-data/ts-2/wal
uuid: "443bda07921549e1b17da4a04c0eadb5"
format_stamp: "Formatted at 2026-08-07 18:12:18 on dist-test-slave-gkw7"
server_key: "b74d63fa20594aae466b4bc3c167ae82"
server_key_iv: "92e8fa3dd1a13512f3b24b62b3b18387"
server_key_version: "encryptionkey@0"
I20260807 18:12:18.395180 20347 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126328808105-19048-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126328808105-19048-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126328808105-19048-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:12:18.407630 20347 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:12:18.412257 20347 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:12:18.412555 20347 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:12:18.413677 20347 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:12:18.415087 20347 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:12:18.415160 20347 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:12:18.415295 20347 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:12:18.415351 20347 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:12:18.449774 20142 heartbeater.cc:503] Master 127.18.154.62:44877 was elected leader, sending a full tablet report...
I20260807 18:12:18.451256 20347 rpc_server.cc:307] RPC server started. Bound to: 127.18.154.3:38203
I20260807 18:12:18.451308 20539 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.154.3:38203 every 8 connection(s)
I20260807 18:12:18.452698 20347 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126328808105-19048-0/minicluster-data/ts-2/data/info.pb
I20260807 18:12:18.462936 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 20347
I20260807 18:12:18.463142 19048 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786126328808105-19048-0/minicluster-data/ts-2/wal/instance
I20260807 18:12:18.463600 19048 external_mini_cluster.cc:1428] Setting key 9d6749d00a7360846c4161e9eb4d84a8
Aug 07 18:12:18 dist-test-slave-gkw7 krb5kdc[19805](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126338, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.3@KRBTEST.COM for kudu/127.18.154.62@KRBTEST.COM
I20260807 18:12:18.474222 20339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:18.455510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.3:40381 (local address 127.18.154.62:44877)
0807 18:12:18.455711 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0807 18:12:18.455716 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:12:18.456848 (+ 1132us) server_negotiation.cc:407] Connection header received
0807 18:12:18.458948 (+ 2100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:12:18.458954 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:12:18.459053 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:12:18.459169 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:12:18.460801 (+ 1632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:18.461656 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:18.463486 (+ 1830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:18.463955 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:18.464093 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:18.468037 (+ 3944us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:12:18.468043 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:12:18.468047 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:12:18.468086 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:12:18.471137 (+ 3051us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:12:18.471952 (+ 815us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:12:18.471956 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:12:18.471959 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:12:18.472089 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:12:18.472505 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:12:18.472508 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:12:18.472510 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:12:18.472833 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:12:18.472960 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0807 18:12:18.473703 (+ 743us) server_negotiation.cc:299] Negotiation successful
0807 18:12:18.473889 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260807 18:12:18.475301 20543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:18.455884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:44877 (local address 127.18.154.3:40381)
0807 18:12:18.456669 (+ 785us) negotiation.cc:107] Waiting for socket to connect
0807 18:12:18.456710 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 18:12:18.458655 (+ 1945us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:12:18.459422 (+ 767us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:12:18.459431 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:12:18.460000 (+ 569us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:12:18.460617 (+ 617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:12:18.460634 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:12:18.461844 (+ 1210us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:12:18.461848 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:12:18.463344 (+ 1496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:12:18.463357 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:12:18.464200 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:12:18.464208 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:12:18.464559 (+ 351us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:18.465427 (+ 868us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:12:18.465451 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:12:18.467818 (+ 2367us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:12:18.471291 (+ 3473us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:12:18.471298 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:12:18.471311 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:12:18.471805 (+ 494us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:12:18.472207 (+ 402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:12:18.472210 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:12:18.472212 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:12:18.472402 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:12:18.472943 (+ 541us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:12:18.472949 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:12:18.473291 (+ 342us) client_negotiation.cc:769] Sending connection context
0807 18:12:18.473523 (+ 232us) client_negotiation.cc:240] Negotiation successful
0807 18:12:18.474039 (+ 516us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":522,"thread_start_us":310,"threads_started":1}
I20260807 18:12:18.476799 20540 heartbeater.cc:347] Connected to a master server at 127.18.154.62:44877
I20260807 18:12:18.477157 20540 heartbeater.cc:464] Registering TS with master...
I20260807 18:12:18.478137 20540 heartbeater.cc:511] Master 127.18.154.62:44877 requested a full tablet report, sending...
I20260807 18:12:18.479957 19867 ts_manager.cc:194] Registered new tserver with Master: 443bda07921549e1b17da4a04c0eadb5 (127.18.154.3:38203)
I20260807 18:12:18.480907 19867 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.154.3@KRBTEST.COM'} at 127.18.154.3:40381
I20260807 18:12:18.492519 19048 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 18:12:18.514575 20339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:18.500409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40232 (local address 127.18.154.62:44877)
0807 18:12:18.500641 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0807 18:12:18.500647 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:12:18.501229 (+ 582us) server_negotiation.cc:407] Connection header received
0807 18:12:18.501358 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:12:18.501362 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:12:18.501445 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:12:18.501613 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:12:18.503200 (+ 1587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:18.503973 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:18.505201 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:18.505704 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:18.505803 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:18.508687 (+ 2884us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:12:18.508693 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:12:18.508697 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:12:18.508765 (+ 68us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:12:18.512010 (+ 3245us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:12:18.512669 (+ 659us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:12:18.512674 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:12:18.512677 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:12:18.512834 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:12:18.513171 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:12:18.513175 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:12:18.513177 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:12:18.513590 (+ 413us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:12:18.513810 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0807 18:12:18.514159 (+ 349us) server_negotiation.cc:299] Negotiation successful
0807 18:12:18.514335 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260807 18:12:18.523579 19867 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40232:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 18:12:18.531927 19867 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:12:18.575728 20560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:18.562308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.1:40819 (local address 127.0.0.1:59826)
0807 18:12:18.565777 (+ 3469us) negotiation.cc:107] Waiting for socket to connect
0807 18:12:18.565806 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0807 18:12:18.565926 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:12:18.567032 (+ 1106us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:12:18.567042 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:12:18.567073 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:12:18.567615 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:12:18.567632 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:12:18.569795 (+ 2163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:12:18.569798 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:12:18.572364 (+ 2566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:12:18.572382 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:12:18.574607 (+ 2225us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:12:18.574613 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:12:18.574838 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:18.574891 (+ 53us) client_negotiation.cc:769] Sending connection context
0807 18:12:18.575042 (+ 151us) client_negotiation.cc:240] Negotiation successful
0807 18:12:18.575256 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3331,"mutex_wait_us":2,"thread_start_us":2338,"threads_started":1}
I20260807 18:12:18.575760 20567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:18.562735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59826 (local address 127.18.154.1:40819)
0807 18:12:18.566244 (+ 3509us) server_negotiation.cc:206] Beginning negotiation
0807 18:12:18.566254 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:12:18.566288 (+ 34us) server_negotiation.cc:407] Connection header received
0807 18:12:18.566410 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:12:18.566421 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:12:18.566752 (+ 331us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:12:18.566989 (+ 237us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:12:18.567912 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:18.569584 (+ 1672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:18.572562 (+ 2978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:18.574387 (+ 1825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:18.574555 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:18.574833 (+ 278us) server_negotiation.cc:1035] Waiting for connection context
0807 18:12:18.575139 (+ 306us) server_negotiation.cc:299] Negotiation successful
0807 18:12:18.575334 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3276,"thread_start_us":3123,"threads_started":1}
I20260807 18:12:18.576900 20561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:18.548524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.2:39287 (local address 127.0.0.1:36110)
0807 18:12:18.559825 (+ 11301us) negotiation.cc:107] Waiting for socket to connect
0807 18:12:18.559920 (+ 95us) client_negotiation.cc:174] Beginning negotiation
0807 18:12:18.560308 (+ 388us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:12:18.567483 (+ 7175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:12:18.567489 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:12:18.567517 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:12:18.568044 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:12:18.568054 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:12:18.571197 (+ 3143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:12:18.571202 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:12:18.573705 (+ 2503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:12:18.573722 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:12:18.575880 (+ 2158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:12:18.575889 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:12:18.576110 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:18.576159 (+ 49us) client_negotiation.cc:769] Sending connection context
0807 18:12:18.576241 (+ 82us) client_negotiation.cc:240] Negotiation successful
0807 18:12:18.576403 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10881,"mutex_wait_us":4,"thread_start_us":16902,"threads_started":1}
I20260807 18:12:18.577011 20565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:18.549035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37610 (local address 127.18.154.3:38203)
0807 18:12:18.566985 (+ 17950us) server_negotiation.cc:206] Beginning negotiation
0807 18:12:18.566994 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:12:18.567027 (+ 33us) server_negotiation.cc:407] Connection header received
0807 18:12:18.567137 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:12:18.567147 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:12:18.567358 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:12:18.567559 (+ 201us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:12:18.568766 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:18.570810 (+ 2044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:18.573979 (+ 3169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:18.575474 (+ 1495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:18.575626 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:18.575933 (+ 307us) server_negotiation.cc:1035] Waiting for connection context
0807 18:12:18.576493 (+ 560us) server_negotiation.cc:299] Negotiation successful
0807 18:12:18.576669 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17749,"thread_start_us":17577,"threads_started":1}
I20260807 18:12:18.577083 20566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:18.548728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.3:38203 (local address 127.0.0.1:37610)
0807 18:12:18.565151 (+ 16423us) negotiation.cc:107] Waiting for socket to connect
0807 18:12:18.565191 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 18:12:18.565371 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:12:18.567556 (+ 2185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:12:18.567561 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:12:18.567627 (+ 66us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:12:18.568437 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:12:18.568444 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:12:18.570977 (+ 2533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:12:18.570983 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:12:18.573821 (+ 2838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:12:18.573836 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:12:18.576098 (+ 2262us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:12:18.576105 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:12:18.576291 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:18.576326 (+ 35us) client_negotiation.cc:769] Sending connection context
0807 18:12:18.576423 (+ 97us) client_negotiation.cc:240] Negotiation successful
0807 18:12:18.576557 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16131,"mutex_wait_us":339,"spinlock_wait_cycles":11520,"thread_start_us":11886,"threads_started":1}
I20260807 18:12:18.581298 20050 tablet_service.cc:1467] Processing CreateTablet for tablet b11220df1b5c446987e645bdc231533f (DEFAULT_TABLE table=test-table [id=2ed6299ca06f4dc39897ef6b58109dd5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:12:18.581290 20436 tablet_service.cc:1467] Processing CreateTablet for tablet b11220df1b5c446987e645bdc231533f (DEFAULT_TABLE table=test-table [id=2ed6299ca06f4dc39897ef6b58109dd5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:12:18.582944 20050 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b11220df1b5c446987e645bdc231533f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:12:18.583633 20564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:18.548739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36110 (local address 127.18.154.2:39287)
0807 18:12:18.566620 (+ 17881us) server_negotiation.cc:206] Beginning negotiation
0807 18:12:18.566630 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:12:18.566664 (+ 34us) server_negotiation.cc:407] Connection header received
0807 18:12:18.566771 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:12:18.566778 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:12:18.567079 (+ 301us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:12:18.568734 (+ 1655us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:12:18.569187 (+ 453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:18.571045 (+ 1858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:18.573929 (+ 2884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:18.575588 (+ 1659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:18.577329 (+ 1741us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:18.577543 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0807 18:12:18.578440 (+ 897us) server_negotiation.cc:299] Negotiation successful
0807 18:12:18.583292 (+ 4852us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17642,"thread_start_us":17442,"threads_started":1}
I20260807 18:12:18.585492 20436 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b11220df1b5c446987e645bdc231533f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:12:18.587280 20236 tablet_service.cc:1467] Processing CreateTablet for tablet b11220df1b5c446987e645bdc231533f (DEFAULT_TABLE table=test-table [id=2ed6299ca06f4dc39897ef6b58109dd5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:12:18.588786 20236 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b11220df1b5c446987e645bdc231533f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:12:18.598978 20568 tablet_bootstrap.cc:492] T b11220df1b5c446987e645bdc231533f P 443bda07921549e1b17da4a04c0eadb5: Bootstrap starting.
I20260807 18:12:18.601351 20569 tablet_bootstrap.cc:492] T b11220df1b5c446987e645bdc231533f P 03217377ae944cd39f87ad5119c21e9d: Bootstrap starting.
I20260807 18:12:18.604739 20570 tablet_bootstrap.cc:492] T b11220df1b5c446987e645bdc231533f P 6f7ed67d931d4357a07d5c38f36d91a3: Bootstrap starting.
I20260807 18:12:18.605309 20569 tablet_bootstrap.cc:654] T b11220df1b5c446987e645bdc231533f P 03217377ae944cd39f87ad5119c21e9d: Neither blocks nor log segments found. Creating new log.
I20260807 18:12:18.605307 20568 tablet_bootstrap.cc:654] T b11220df1b5c446987e645bdc231533f P 443bda07921549e1b17da4a04c0eadb5: Neither blocks nor log segments found. Creating new log.
I20260807 18:12:18.606897 20569 log.cc:824] T b11220df1b5c446987e645bdc231533f P 03217377ae944cd39f87ad5119c21e9d: Log is configured to *not* fsync() on all Append() calls
I20260807 18:12:18.606999 20568 log.cc:824] T b11220df1b5c446987e645bdc231533f P 443bda07921549e1b17da4a04c0eadb5: Log is configured to *not* fsync() on all Append() calls
I20260807 18:12:18.607954 20570 tablet_bootstrap.cc:654] T b11220df1b5c446987e645bdc231533f P 6f7ed67d931d4357a07d5c38f36d91a3: Neither blocks nor log segments found. Creating new log.
I20260807 18:12:18.609377 20570 log.cc:824] T b11220df1b5c446987e645bdc231533f P 6f7ed67d931d4357a07d5c38f36d91a3: Log is configured to *not* fsync() on all Append() calls
I20260807 18:12:18.610904 20569 tablet_bootstrap.cc:492] T b11220df1b5c446987e645bdc231533f P 03217377ae944cd39f87ad5119c21e9d: No bootstrap required, opened a new log
I20260807 18:12:18.610924 20568 tablet_bootstrap.cc:492] T b11220df1b5c446987e645bdc231533f P 443bda07921549e1b17da4a04c0eadb5: No bootstrap required, opened a new log
I20260807 18:12:18.611223 20568 ts_tablet_manager.cc:1397] T b11220df1b5c446987e645bdc231533f P 443bda07921549e1b17da4a04c0eadb5: Time spent bootstrapping tablet: real 0.013s user 0.008s sys 0.000s
I20260807 18:12:18.611236 20569 ts_tablet_manager.cc:1397] T b11220df1b5c446987e645bdc231533f P 03217377ae944cd39f87ad5119c21e9d: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260807 18:12:18.612273 20570 tablet_bootstrap.cc:492] T b11220df1b5c446987e645bdc231533f P 6f7ed67d931d4357a07d5c38f36d91a3: No bootstrap required, opened a new log
I20260807 18:12:18.612545 20570 ts_tablet_manager.cc:1397] T b11220df1b5c446987e645bdc231533f P 6f7ed67d931d4357a07d5c38f36d91a3: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260807 18:12:18.620731 20570 raft_consensus.cc:348] T b11220df1b5c446987e645bdc231533f P 6f7ed67d931d4357a07d5c38f36d91a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f7ed67d931d4357a07d5c38f36d91a3" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 39287 } } peers { permanent_uuid: "443bda07921549e1b17da4a04c0eadb5" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 38203 } } peers { permanent_uuid: "03217377ae944cd39f87ad5119c21e9d" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 40819 } }
I20260807 18:12:18.621215 20570 raft_consensus.cc:374] T b11220df1b5c446987e645bdc231533f P 6f7ed67d931d4357a07d5c38f36d91a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:12:18.621337 20570 raft_consensus.cc:729] T b11220df1b5c446987e645bdc231533f P 6f7ed67d931d4357a07d5c38f36d91a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f7ed67d931d4357a07d5c38f36d91a3, State: Initialized, Role: FOLLOWER
I20260807 18:12:18.622131 20570 consensus_queue.cc:260] T b11220df1b5c446987e645bdc231533f P 6f7ed67d931d4357a07d5c38f36d91a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f7ed67d931d4357a07d5c38f36d91a3" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 39287 } } peers { permanent_uuid: "443bda07921549e1b17da4a04c0eadb5" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 38203 } } peers { permanent_uuid: "03217377ae944cd39f87ad5119c21e9d" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 40819 } }
I20260807 18:12:18.623611 20335 heartbeater.cc:503] Master 127.18.154.62:44877 was elected leader, sending a full tablet report...
I20260807 18:12:18.623737 20569 raft_consensus.cc:348] T b11220df1b5c446987e645bdc231533f P 03217377ae944cd39f87ad5119c21e9d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f7ed67d931d4357a07d5c38f36d91a3" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 39287 } } peers { permanent_uuid: "443bda07921549e1b17da4a04c0eadb5" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 38203 } } peers { permanent_uuid: "03217377ae944cd39f87ad5119c21e9d" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 40819 } }
I20260807 18:12:18.624198 20569 raft_consensus.cc:374] T b11220df1b5c446987e645bdc231533f P 03217377ae944cd39f87ad5119c21e9d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:12:18.624317 20569 raft_consensus.cc:729] T b11220df1b5c446987e645bdc231533f P 03217377ae944cd39f87ad5119c21e9d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03217377ae944cd39f87ad5119c21e9d, State: Initialized, Role: FOLLOWER
I20260807 18:12:18.624370 20570 ts_tablet_manager.cc:1428] T b11220df1b5c446987e645bdc231533f P 6f7ed67d931d4357a07d5c38f36d91a3: Time spent starting tablet: real 0.012s user 0.012s sys 0.000s
I20260807 18:12:18.625029 20569 consensus_queue.cc:260] T b11220df1b5c446987e645bdc231533f P 03217377ae944cd39f87ad5119c21e9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f7ed67d931d4357a07d5c38f36d91a3" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 39287 } } peers { permanent_uuid: "443bda07921549e1b17da4a04c0eadb5" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 38203 } } peers { permanent_uuid: "03217377ae944cd39f87ad5119c21e9d" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 40819 } }
I20260807 18:12:18.625408 20568 raft_consensus.cc:348] T b11220df1b5c446987e645bdc231533f P 443bda07921549e1b17da4a04c0eadb5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f7ed67d931d4357a07d5c38f36d91a3" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 39287 } } peers { permanent_uuid: "443bda07921549e1b17da4a04c0eadb5" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 38203 } } peers { permanent_uuid: "03217377ae944cd39f87ad5119c21e9d" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 40819 } }
I20260807 18:12:18.625933 20568 raft_consensus.cc:374] T b11220df1b5c446987e645bdc231533f P 443bda07921549e1b17da4a04c0eadb5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:12:18.626046 20568 raft_consensus.cc:729] T b11220df1b5c446987e645bdc231533f P 443bda07921549e1b17da4a04c0eadb5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 443bda07921549e1b17da4a04c0eadb5, State: Initialized, Role: FOLLOWER
I20260807 18:12:18.626700 20568 consensus_queue.cc:260] T b11220df1b5c446987e645bdc231533f P 443bda07921549e1b17da4a04c0eadb5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f7ed67d931d4357a07d5c38f36d91a3" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 39287 } } peers { permanent_uuid: "443bda07921549e1b17da4a04c0eadb5" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 38203 } } peers { permanent_uuid: "03217377ae944cd39f87ad5119c21e9d" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 40819 } }
I20260807 18:12:18.628388 20540 heartbeater.cc:503] Master 127.18.154.62:44877 was elected leader, sending a full tablet report...
I20260807 18:12:18.629070 20568 ts_tablet_manager.cc:1428] T b11220df1b5c446987e645bdc231533f P 443bda07921549e1b17da4a04c0eadb5: Time spent starting tablet: real 0.015s user 0.014s sys 0.000s
I20260807 18:12:18.630474 20569 ts_tablet_manager.cc:1428] T b11220df1b5c446987e645bdc231533f P 03217377ae944cd39f87ad5119c21e9d: Time spent starting tablet: real 0.019s user 0.009s sys 0.005s
W20260807 18:12:18.668056 20143 tablet.cc:2369] T b11220df1b5c446987e645bdc231533f P 03217377ae944cd39f87ad5119c21e9d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:12:18.683116 20582 raft_consensus.cc:482] T b11220df1b5c446987e645bdc231533f P 443bda07921549e1b17da4a04c0eadb5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:12:18.683508 20582 raft_consensus.cc:504] T b11220df1b5c446987e645bdc231533f P 443bda07921549e1b17da4a04c0eadb5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f7ed67d931d4357a07d5c38f36d91a3" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 39287 } } peers { permanent_uuid: "443bda07921549e1b17da4a04c0eadb5" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 38203 } } peers { permanent_uuid: "03217377ae944cd39f87ad5119c21e9d" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 40819 } }
I20260807 18:12:18.685840 20582 leader_election.cc:288] T b11220df1b5c446987e645bdc231533f P 443bda07921549e1b17da4a04c0eadb5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f7ed67d931d4357a07d5c38f36d91a3 (127.18.154.2:39287), 03217377ae944cd39f87ad5119c21e9d (127.18.154.1:40819)
W20260807 18:12:18.689333 20337 tablet.cc:2369] T b11220df1b5c446987e645bdc231533f P 6f7ed67d931d4357a07d5c38f36d91a3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:12:18.692828 20543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:18.686361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.1:40819 (local address 127.18.154.3:58893)
0807 18:12:18.686543 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0807 18:12:18.686564 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 18:12:18.686692 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:12:18.687065 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:12:18.687072 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:12:18.687108 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:12:18.687514 (+ 406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:12:18.687534 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:12:18.688819 (+ 1285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:12:18.688832 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:12:18.690684 (+ 1852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:12:18.690700 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:12:18.692033 (+ 1333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:12:18.692039 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:12:18.692246 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:18.692281 (+ 35us) client_negotiation.cc:769] Sending connection context
0807 18:12:18.692386 (+ 105us) client_negotiation.cc:240] Negotiation successful
0807 18:12:18.692525 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260807 18:12:18.692842 20567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:18.686405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.3:58893 (local address 127.18.154.1:40819)
0807 18:12:18.686615 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0807 18:12:18.686625 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:12:18.686644 (+ 19us) server_negotiation.cc:407] Connection header received
0807 18:12:18.686830 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:12:18.686834 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:12:18.686898 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:12:18.686990 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:12:18.687735 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:18.688653 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:18.690866 (+ 2213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:18.691863 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:18.692022 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:18.692108 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0807 18:12:18.692440 (+ 332us) server_negotiation.cc:299] Negotiation successful
0807 18:12:18.692585 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 18:12:18.693920 20583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:18.686361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.2:39287 (local address 127.18.154.3:33557)
0807 18:12:18.687747 (+ 1386us) negotiation.cc:107] Waiting for socket to connect
0807 18:12:18.687762 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 18:12:18.688015 (+ 253us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:12:18.688382 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:12:18.688386 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:12:18.688414 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:12:18.688739 (+ 325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:12:18.688748 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:12:18.689982 (+ 1234us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:12:18.689991 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:12:18.691383 (+ 1392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:12:18.691395 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:12:18.693079 (+ 1684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:12:18.693086 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:12:18.693260 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:18.693297 (+ 37us) client_negotiation.cc:769] Sending connection context
0807 18:12:18.693403 (+ 106us) client_negotiation.cc:240] Negotiation successful
0807 18:12:18.693536 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1275,"mutex_wait_us":409,"thread_start_us":605,"threads_started":1}
I20260807 18:12:18.694264 20564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:18.686323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.3:33557 (local address 127.18.154.2:39287)
0807 18:12:18.686559 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0807 18:12:18.686567 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:12:18.687963 (+ 1396us) server_negotiation.cc:407] Connection header received
0807 18:12:18.688177 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:12:18.688181 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:12:18.688263 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:12:18.688360 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:12:18.688912 (+ 552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:18.689828 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:18.691553 (+ 1725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:18.692759 (+ 1206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:18.692922 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:18.693016 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0807 18:12:18.693844 (+ 828us) server_negotiation.cc:299] Negotiation successful
0807 18:12:18.694016 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260807 18:12:18.694541 20083 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b11220df1b5c446987e645bdc231533f" candidate_uuid: "443bda07921549e1b17da4a04c0eadb5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03217377ae944cd39f87ad5119c21e9d" is_pre_election: true
I20260807 18:12:18.695214 20083 raft_consensus.cc:2454] T b11220df1b5c446987e645bdc231533f P 03217377ae944cd39f87ad5119c21e9d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 443bda07921549e1b17da4a04c0eadb5 in term 0.
I20260807 18:12:18.696095 20368 leader_election.cc:302] T b11220df1b5c446987e645bdc231533f P 443bda07921549e1b17da4a04c0eadb5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03217377ae944cd39f87ad5119c21e9d, 443bda07921549e1b17da4a04c0eadb5; no voters:
I20260807 18:12:18.696554 20582 raft_consensus.cc:2781] T b11220df1b5c446987e645bdc231533f P 443bda07921549e1b17da4a04c0eadb5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:12:18.696738 20582 raft_consensus.cc:482] T b11220df1b5c446987e645bdc231533f P 443bda07921549e1b17da4a04c0eadb5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:12:18.696520 20263 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b11220df1b5c446987e645bdc231533f" candidate_uuid: "443bda07921549e1b17da4a04c0eadb5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f7ed67d931d4357a07d5c38f36d91a3" is_pre_election: true
I20260807 18:12:18.696877 20582 raft_consensus.cc:3037] T b11220df1b5c446987e645bdc231533f P 443bda07921549e1b17da4a04c0eadb5 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:12:18.697117 20263 raft_consensus.cc:2454] T b11220df1b5c446987e645bdc231533f P 6f7ed67d931d4357a07d5c38f36d91a3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 443bda07921549e1b17da4a04c0eadb5 in term 0.
I20260807 18:12:18.699503 20582 raft_consensus.cc:504] T b11220df1b5c446987e645bdc231533f P 443bda07921549e1b17da4a04c0eadb5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f7ed67d931d4357a07d5c38f36d91a3" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 39287 } } peers { permanent_uuid: "443bda07921549e1b17da4a04c0eadb5" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 38203 } } peers { permanent_uuid: "03217377ae944cd39f87ad5119c21e9d" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 40819 } }
I20260807 18:12:18.700157 20582 leader_election.cc:288] T b11220df1b5c446987e645bdc231533f P 443bda07921549e1b17da4a04c0eadb5 [CANDIDATE]: Term 1 election: Requested vote from peers 6f7ed67d931d4357a07d5c38f36d91a3 (127.18.154.2:39287), 03217377ae944cd39f87ad5119c21e9d (127.18.154.1:40819)
I20260807 18:12:18.700598 20263 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b11220df1b5c446987e645bdc231533f" candidate_uuid: "443bda07921549e1b17da4a04c0eadb5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f7ed67d931d4357a07d5c38f36d91a3"
I20260807 18:12:18.700640 20083 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b11220df1b5c446987e645bdc231533f" candidate_uuid: "443bda07921549e1b17da4a04c0eadb5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03217377ae944cd39f87ad5119c21e9d"
I20260807 18:12:18.700822 20263 raft_consensus.cc:3037] T b11220df1b5c446987e645bdc231533f P 6f7ed67d931d4357a07d5c38f36d91a3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:12:18.700835 20083 raft_consensus.cc:3037] T b11220df1b5c446987e645bdc231533f P 03217377ae944cd39f87ad5119c21e9d [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:12:18.703308 20263 raft_consensus.cc:2454] T b11220df1b5c446987e645bdc231533f P 6f7ed67d931d4357a07d5c38f36d91a3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 443bda07921549e1b17da4a04c0eadb5 in term 1.
I20260807 18:12:18.703518 20083 raft_consensus.cc:2454] T b11220df1b5c446987e645bdc231533f P 03217377ae944cd39f87ad5119c21e9d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 443bda07921549e1b17da4a04c0eadb5 in term 1.
I20260807 18:12:18.703838 20372 leader_election.cc:302] T b11220df1b5c446987e645bdc231533f P 443bda07921549e1b17da4a04c0eadb5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 443bda07921549e1b17da4a04c0eadb5, 6f7ed67d931d4357a07d5c38f36d91a3; no voters:
I20260807 18:12:18.704185 20582 raft_consensus.cc:2781] T b11220df1b5c446987e645bdc231533f P 443bda07921549e1b17da4a04c0eadb5 [term 1 FOLLOWER]: Leader election won for term 1
W20260807 18:12:18.704351 20541 tablet.cc:2369] T b11220df1b5c446987e645bdc231533f P 443bda07921549e1b17da4a04c0eadb5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:12:18.704764 20582 raft_consensus.cc:686] T b11220df1b5c446987e645bdc231533f P 443bda07921549e1b17da4a04c0eadb5 [term 1 LEADER]: Becoming Leader. State: Replica: 443bda07921549e1b17da4a04c0eadb5, State: Running, Role: LEADER
I20260807 18:12:18.705581 20582 consensus_queue.cc:237] T b11220df1b5c446987e645bdc231533f P 443bda07921549e1b17da4a04c0eadb5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f7ed67d931d4357a07d5c38f36d91a3" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 39287 } } peers { permanent_uuid: "443bda07921549e1b17da4a04c0eadb5" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 38203 } } peers { permanent_uuid: "03217377ae944cd39f87ad5119c21e9d" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 40819 } }
I20260807 18:12:18.711372 19867 catalog_manager.cc:5495] T b11220df1b5c446987e645bdc231533f P 443bda07921549e1b17da4a04c0eadb5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 443bda07921549e1b17da4a04c0eadb5 (127.18.154.3). New cstate: current_term: 1 leader_uuid: "443bda07921549e1b17da4a04c0eadb5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f7ed67d931d4357a07d5c38f36d91a3" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 39287 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "443bda07921549e1b17da4a04c0eadb5" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 38203 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "03217377ae944cd39f87ad5119c21e9d" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 40819 } health_report { overall_health: UNKNOWN } } }
Aug 07 18:12:18 dist-test-slave-gkw7 krb5kdc[19805](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126336, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.154.1@KRBTEST.COM
I20260807 18:12:18.742288 20567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:18.725663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59834 (local address 127.18.154.1:40819)
0807 18:12:18.725862 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0807 18:12:18.725867 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:12:18.726496 (+ 629us) server_negotiation.cc:407] Connection header received
0807 18:12:18.726712 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:12:18.726717 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:12:18.726788 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:12:18.726903 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:12:18.728454 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:18.729254 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:18.730613 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:18.731066 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:18.731209 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:18.735485 (+ 4276us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:12:18.735502 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:12:18.735521 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:12:18.735563 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:12:18.738989 (+ 3426us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:12:18.739882 (+ 893us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:12:18.739890 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:12:18.739893 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:12:18.740039 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:12:18.740526 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:12:18.740530 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:12:18.740532 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:12:18.740911 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:12:18.741181 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0807 18:12:18.741598 (+ 417us) server_negotiation.cc:299] Negotiation successful
0807 18:12:18.741799 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
Aug 07 18:12:18 dist-test-slave-gkw7 krb5kdc[19805](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126338, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 18:12:18 dist-test-slave-gkw7 krb5kdc[19805](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126338, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.18.154.1@KRBTEST.COM
I20260807 18:12:18.801046 20567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:18.786100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59846 (local address 127.18.154.1:40819)
0807 18:12:18.786337 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0807 18:12:18.786348 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 18:12:18.786769 (+ 421us) server_negotiation.cc:407] Connection header received
0807 18:12:18.787002 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:12:18.787006 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:12:18.787073 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:12:18.787183 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:12:18.788689 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:18.789472 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:18.790803 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:18.791217 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:18.791363 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:18.795483 (+ 4120us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:12:18.795489 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:12:18.795492 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:12:18.795530 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:12:18.798703 (+ 3173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:12:18.799383 (+ 680us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:12:18.799389 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:12:18.799391 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:12:18.799534 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:12:18.799902 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:12:18.799905 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:12:18.799907 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:12:18.800233 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:12:18.800425 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0807 18:12:18.800627 (+ 202us) server_negotiation.cc:299] Negotiation successful
0807 18:12:18.800795 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
Aug 07 18:12:18 dist-test-slave-gkw7 krb5kdc[19805](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126338, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 18:12:18 dist-test-slave-gkw7 krb5kdc[19805](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126338, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.154.1@KRBTEST.COM
I20260807 18:12:18.851892 20567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:18.837898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59848 (local address 127.18.154.1:40819)
0807 18:12:18.838164 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0807 18:12:18.838172 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:12:18.838490 (+ 318us) server_negotiation.cc:407] Connection header received
0807 18:12:18.838646 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:12:18.838650 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:12:18.838731 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:12:18.838818 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:12:18.840208 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:18.840917 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:18.842329 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:18.842761 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:18.842855 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:18.846969 (+ 4114us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:12:18.846974 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:12:18.846978 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:12:18.847024 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:12:18.849420 (+ 2396us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:12:18.850085 (+ 665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:12:18.850088 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:12:18.850091 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:12:18.850209 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:12:18.850531 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:12:18.850534 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:12:18.850536 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:12:18.850902 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:12:18.851083 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0807 18:12:18.851339 (+ 256us) server_negotiation.cc:299] Negotiation successful
0807 18:12:18.851517 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
W20260807 18:12:18.854295 20021 tablet_service.cc:2908] Rejecting scan request for tablet b11220df1b5c446987e645bdc231533f: Uninitialized: clean time has not yet been initialized
I20260807 18:12:18.858717 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 19956
I20260807 18:12:18.873081 20567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:18.859691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59850 (local address 127.18.154.1:40819)
0807 18:12:18.859941 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0807 18:12:18.859952 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 18:12:18.860341 (+ 389us) server_negotiation.cc:407] Connection header received
0807 18:12:18.860612 (+ 271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:12:18.860619 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:12:18.860723 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:12:18.860855 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:12:18.862381 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:18.863191 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:18.864702 (+ 1511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:18.865229 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:18.865406 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:18.867889 (+ 2483us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:12:18.867894 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:12:18.867897 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:12:18.867930 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:12:18.870758 (+ 2828us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:12:18.871454 (+ 696us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:12:18.871457 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:12:18.871460 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:12:18.871576 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:12:18.871909 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:12:18.871912 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:12:18.871914 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:12:18.872162 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:12:18.872374 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0807 18:12:18.872645 (+ 271us) server_negotiation.cc:299] Negotiation successful
0807 18:12:18.872838 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260807 18:12:18.873941 20135 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:12:18.970800 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 19956
I20260807 18:12:18.999215 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 20151
Aug 07 18:12:19 dist-test-slave-gkw7 krb5kdc[19805](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126338, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.154.2@KRBTEST.COM
I20260807 18:12:19.014920 20564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:19.000084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36122 (local address 127.18.154.2:39287)
0807 18:12:19.000276 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0807 18:12:19.000280 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:12:19.000300 (+ 20us) server_negotiation.cc:407] Connection header received
0807 18:12:19.000446 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:12:19.000451 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:12:19.000517 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:12:19.000603 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:12:19.001894 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:19.002733 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:19.004051 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:19.004457 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:19.004572 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:19.008933 (+ 4361us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:12:19.008949 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:12:19.008966 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:12:19.009010 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:12:19.012114 (+ 3104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:12:19.012793 (+ 679us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:12:19.012800 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:12:19.012803 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:12:19.012934 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:12:19.013401 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:12:19.013404 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:12:19.013406 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:12:19.013871 (+ 465us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:12:19.014095 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0807 18:12:19.014447 (+ 352us) server_negotiation.cc:299] Negotiation successful
0807 18:12:19.014617 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 18:12:19.015691 20327 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:12:19.106403 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 20151
I20260807 18:12:19.129644 20583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:19.128840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.1:40819 (local address 127.18.154.3:46389)
0807 18:12:19.129110 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0807 18:12:19.129332 (+ 222us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.1:40819: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
W20260807 18:12:19.130023 20368 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.154.1:40819: connect: Connection refused (error 111)
I20260807 18:12:19.130764 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 20347
I20260807 18:12:19.131811 20583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:19.131348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.1:40819 (local address 127.18.154.3:53427)
0807 18:12:19.131513 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0807 18:12:19.131623 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.1:40819: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260807 18:12:19.132104 20368 consensus_peers.cc:487] T b11220df1b5c446987e645bdc231533f P 443bda07921549e1b17da4a04c0eadb5 -> Peer 03217377ae944cd39f87ad5119c21e9d (127.18.154.1:40819): Couldn't send request to peer 03217377ae944cd39f87ad5119c21e9d. Status: Network error: Client connection negotiation failed: client connection to 127.18.154.1:40819: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
Aug 07 18:12:19 dist-test-slave-gkw7 krb5kdc[19805](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126338, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.154.3@KRBTEST.COM
I20260807 18:12:19.146718 20606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:19.131539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37622 (local address 127.18.154.3:38203)
0807 18:12:19.132123 (+ 584us) server_negotiation.cc:206] Beginning negotiation
0807 18:12:19.132128 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:12:19.132152 (+ 24us) server_negotiation.cc:407] Connection header received
0807 18:12:19.132422 (+ 270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:12:19.132427 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:12:19.132494 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:12:19.132600 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:12:19.133866 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:19.134564 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:19.135869 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:19.136287 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:19.136389 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:19.140607 (+ 4218us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:12:19.140619 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:12:19.140637 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:12:19.140671 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:12:19.143782 (+ 3111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:12:19.144405 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:12:19.144413 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:12:19.144416 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:12:19.144563 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:12:19.144946 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:12:19.144951 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:12:19.144954 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:12:19.145397 (+ 443us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:12:19.145764 (+ 367us) server_negotiation.cc:1035] Waiting for connection context
0807 18:12:19.145959 (+ 195us) server_negotiation.cc:299] Negotiation successful
0807 18:12:19.146380 (+ 421us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":395,"thread_start_us":280,"threads_started":1}
I20260807 18:12:19.147446 20533 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:12:19.239018 20583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:19.238519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.2:39287 (local address 127.18.154.3:47575)
0807 18:12:19.238747 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0807 18:12:19.238835 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.2:39287: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260807 18:12:19.239037 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 20347
I20260807 18:12:19.262522 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 19822
I20260807 18:12:19.263813 19910 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:12:19.376730 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 19822
2026-08-07T18:12:19Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (4657 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126328808105-19048-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:12:19 dist-test-slave-gkw7 krb5kdc[20611](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:12:19 dist-test-slave-gkw7 krb5kdc[20611](info): set up 2 sockets
Aug 07 18:12:19 dist-test-slave-gkw7 krb5kdc[20611](info): commencing operation
krb5kdc: starting...
W20260807 18:12:20.650305 19048 mini_kdc.cc:121] Time spent starting KDC: real 1.223s user 0.004s sys 0.016s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126328808105-19048-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126328808105-19048-0/krb5kdc/test-user.keytab.
Aug 07 18:12:20 dist-test-slave-gkw7 krb5kdc[20611](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126340, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:12:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:12:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.154.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.154.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.154.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.62.keytab.
Entry for principal kudu/127.18.154.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.62.keytab.
I20260807 18:12:20.866109 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126328808105-19048-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126328808105-19048-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126328808105-19048-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.154.62:40323
--webserver_interface=127.18.154.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.62.keytab
--principal=kudu/127.18.154.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.154.20:42303
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.154.62:40323
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126328808105-19048-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126328808105-19048-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126328808105-19048-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:12:21.209926 20635 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:12:21.210240 20635 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:12:21.210332 20635 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:12:21.217464 20635 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:12:21.217650 20635 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260807 18:12:21.217724 20635 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:12:21.217790 20635 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:12:21.217845 20635 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:12:21.217908 20635 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:12:21.228238 20635 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.154.20:42303
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126328808105-19048-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126328808105-19048-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.154.62:40323
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.62.keytab
--principal=kudu/127.18.154.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.154.62:40323
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.154.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126328808105-19048-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:12:21.229611 20635 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:12:21.230995 20635 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:12:21.243474 20641 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:12:21.243513 20646 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:12:21.243561 20643 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:12:21.245112 20635 server_base.cc:1034] running on GCE node
I20260807 18:12:21.246052 20635 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:12:21.247694 20635 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:12:21.248947 20635 hybrid_clock.cc:648] HybridClock initialized: now 1786126341248838 us; error 81 us; skew 500 ppm
I20260807 18:12:21.249393 20635 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:12:21 dist-test-slave-gkw7 krb5kdc[20611](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126341, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:12:21.253065 20635 init.cc:377] Logged in from keytab as kudu/127.18.154.62@KRBTEST.COM (short username kudu)
I20260807 18:12:21.254717 20635 webserver.cc:460] Webserver started at http://127.18.154.62:33909/ using document root <none> and password file <none>
I20260807 18:12:21.255486 20635 fs_manager.cc:359] Metadata directory not provided
I20260807 18:12:21.255585 20635 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:12:21.255874 20635 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:12:21.259300 20635 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126328808105-19048-0/minicluster-data/master-0/data/instance:
uuid: "98661d04c87d4fd0834cfb4532531d75"
format_stamp: "Formatted at 2026-08-07 18:12:21 on dist-test-slave-gkw7"
server_key: "91bdf9065ad13b483c8ae974b75dfe5c"
server_key_iv: "b4af965badab1b79ee14068113a71917"
server_key_version: "encryptionkey@0"
I20260807 18:12:21.260126 20635 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126328808105-19048-0/minicluster-data/master-0/wal/instance:
uuid: "98661d04c87d4fd0834cfb4532531d75"
format_stamp: "Formatted at 2026-08-07 18:12:21 on dist-test-slave-gkw7"
server_key: "91bdf9065ad13b483c8ae974b75dfe5c"
server_key_iv: "b4af965badab1b79ee14068113a71917"
server_key_version: "encryptionkey@0"
I20260807 18:12:21.266271 20635 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.005s
I20260807 18:12:21.270555 20655 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:12:21.272213 20635 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.003s
I20260807 18:12:21.272449 20635 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126328808105-19048-0/minicluster-data/master-0/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126328808105-19048-0/minicluster-data/master-0/wal
uuid: "98661d04c87d4fd0834cfb4532531d75"
format_stamp: "Formatted at 2026-08-07 18:12:21 on dist-test-slave-gkw7"
server_key: "91bdf9065ad13b483c8ae974b75dfe5c"
server_key_iv: "b4af965badab1b79ee14068113a71917"
server_key_version: "encryptionkey@0"
I20260807 18:12:21.272754 20635 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126328808105-19048-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126328808105-19048-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126328808105-19048-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:12:21.306273 20635 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:12:21.310889 20635 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:12:21.311156 20635 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:12:21.335980 20635 rpc_server.cc:307] RPC server started. Bound to: 127.18.154.62:40323
I20260807 18:12:21.335988 20757 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.154.62:40323 every 8 connection(s)
I20260807 18:12:21.337666 20635 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
I20260807 18:12:21.338994 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 20635
I20260807 18:12:21.339236 19048 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126328808105-19048-0/minicluster-data/master-0/wal/instance
I20260807 18:12:21.339991 19048 external_mini_cluster.cc:1428] Setting key bb97d32c70fb116216a0c35e9d77d476
I20260807 18:12:21.342401 20758 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 07 18:12:21 dist-test-slave-gkw7 krb5kdc[20611](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126340, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.154.62@KRBTEST.COM
I20260807 18:12:21.356016 20758 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 98661d04c87d4fd0834cfb4532531d75: Bootstrap starting.
I20260807 18:12:21.360286 20758 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 98661d04c87d4fd0834cfb4532531d75: Neither blocks nor log segments found. Creating new log.
I20260807 18:12:21.361703 20758 log.cc:824] T 00000000000000000000000000000000 P 98661d04c87d4fd0834cfb4532531d75: Log is configured to *not* fsync() on all Append() calls
I20260807 18:12:21.365309 20758 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 98661d04c87d4fd0834cfb4532531d75: No bootstrap required, opened a new log
I20260807 18:12:21.366344 20763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:21.342259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43092 (local address 127.18.154.62:40323)
0807 18:12:21.343180 (+ 921us) server_negotiation.cc:206] Beginning negotiation
0807 18:12:21.343196 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 18:12:21.343247 (+ 51us) server_negotiation.cc:407] Connection header received
0807 18:12:21.345539 (+ 2292us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:12:21.345590 (+ 51us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:12:21.346149 (+ 559us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:12:21.346627 (+ 478us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:12:21.348002 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:21.350155 (+ 2153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:21.351574 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:21.352675 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:21.352891 (+ 216us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:21.357105 (+ 4214us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:12:21.357118 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:12:21.357137 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:12:21.357187 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:12:21.361371 (+ 4184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:12:21.362184 (+ 813us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:12:21.362194 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:12:21.362206 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:12:21.362471 (+ 265us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:12:21.362953 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:12:21.362957 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:12:21.362959 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:12:21.363674 (+ 715us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:12:21.363978 (+ 304us) server_negotiation.cc:1035] Waiting for connection context
0807 18:12:21.364423 (+ 445us) server_negotiation.cc:299] Negotiation successful
0807 18:12:21.364883 (+ 460us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":677,"thread_start_us":442,"threads_started":1}
I20260807 18:12:21.373835 20758 raft_consensus.cc:348] T 00000000000000000000000000000000 P 98661d04c87d4fd0834cfb4532531d75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98661d04c87d4fd0834cfb4532531d75" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 40323 } }
I20260807 18:12:21.374163 20758 raft_consensus.cc:374] T 00000000000000000000000000000000 P 98661d04c87d4fd0834cfb4532531d75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:12:21.374248 20758 raft_consensus.cc:729] T 00000000000000000000000000000000 P 98661d04c87d4fd0834cfb4532531d75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98661d04c87d4fd0834cfb4532531d75, State: Initialized, Role: FOLLOWER
I20260807 18:12:21.374730 20758 consensus_queue.cc:260] T 00000000000000000000000000000000 P 98661d04c87d4fd0834cfb4532531d75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98661d04c87d4fd0834cfb4532531d75" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 40323 } }
I20260807 18:12:21.374976 20758 raft_consensus.cc:388] T 00000000000000000000000000000000 P 98661d04c87d4fd0834cfb4532531d75 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:12:21.375087 20758 raft_consensus.cc:482] T 00000000000000000000000000000000 P 98661d04c87d4fd0834cfb4532531d75 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:12:21.375206 20758 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 98661d04c87d4fd0834cfb4532531d75 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:12:21.377813 20758 raft_consensus.cc:504] T 00000000000000000000000000000000 P 98661d04c87d4fd0834cfb4532531d75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98661d04c87d4fd0834cfb4532531d75" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 40323 } }
I20260807 18:12:21.378290 20758 leader_election.cc:302] T 00000000000000000000000000000000 P 98661d04c87d4fd0834cfb4532531d75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 98661d04c87d4fd0834cfb4532531d75; no voters:
I20260807 18:12:21.378959 20758 leader_election.cc:288] T 00000000000000000000000000000000 P 98661d04c87d4fd0834cfb4532531d75 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:12:21.379098 20765 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 98661d04c87d4fd0834cfb4532531d75 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:12:21.379985 20765 raft_consensus.cc:686] T 00000000000000000000000000000000 P 98661d04c87d4fd0834cfb4532531d75 [term 1 LEADER]: Becoming Leader. State: Replica: 98661d04c87d4fd0834cfb4532531d75, State: Running, Role: LEADER
I20260807 18:12:21.380632 20765 consensus_queue.cc:237] T 00000000000000000000000000000000 P 98661d04c87d4fd0834cfb4532531d75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98661d04c87d4fd0834cfb4532531d75" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 40323 } }
I20260807 18:12:21.381222 20758 sys_catalog.cc:564] T 00000000000000000000000000000000 P 98661d04c87d4fd0834cfb4532531d75 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:12:21.384593 20766 sys_catalog.cc:455] T 00000000000000000000000000000000 P 98661d04c87d4fd0834cfb4532531d75 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "98661d04c87d4fd0834cfb4532531d75" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98661d04c87d4fd0834cfb4532531d75" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 40323 } } }
I20260807 18:12:21.384954 20766 sys_catalog.cc:458] T 00000000000000000000000000000000 P 98661d04c87d4fd0834cfb4532531d75 [sys.catalog]: This master's current role is: LEADER
I20260807 18:12:21.384205 20768 sys_catalog.cc:455] T 00000000000000000000000000000000 P 98661d04c87d4fd0834cfb4532531d75 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 98661d04c87d4fd0834cfb4532531d75. Latest consensus state: current_term: 1 leader_uuid: "98661d04c87d4fd0834cfb4532531d75" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98661d04c87d4fd0834cfb4532531d75" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 40323 } } }
I20260807 18:12:21.385406 20768 sys_catalog.cc:458] T 00000000000000000000000000000000 P 98661d04c87d4fd0834cfb4532531d75 [sys.catalog]: This master's current role is: LEADER
I20260807 18:12:21.386883 20776 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:12:21.392041 20776 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:12:21.401697 20776 catalog_manager.cc:1347] Generated new cluster ID: b097d2c8867c476caaa66d4244385a26
I20260807 18:12:21.401893 20776 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:12:21.412942 20776 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:12:21.414984 20776 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:12:21.434929 20776 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 98661d04c87d4fd0834cfb4532531d75: Generated new TSK 0
I20260807 18:12:21.436139 20776 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.154.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.154.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.154.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.1.keytab.
Entry for principal kudu/127.18.154.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.1.keytab.
I20260807 18:12:21.493281 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126328808105-19048-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126328808105-19048-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126328808105-19048-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126328808105-19048-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.154.1:0
--local_ip_for_outbound_sockets=127.18.154.1
--webserver_interface=127.18.154.1
--webserver_port=0
--tserver_master_addrs=127.18.154.62:40323
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.1.keytab
--principal=kudu/127.18.154.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.154.20:42303
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126328808105-19048-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126328808105-19048-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126328808105-19048-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:12:21.838562 20791 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:12:21.838927 20791 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:12:21.839113 20791 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:12:21.846740 20791 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:12:21.846909 20791 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:12:21.847071 20791 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 18:12:21.847223 20791 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.154.1
I20260807 18:12:21.857517 20791 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.154.20:42303
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126328808105-19048-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126328808105-19048-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.1.keytab
--principal=kudu/127.18.154.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.154.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126328808105-19048-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.154.1
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.18.154.62:40323
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.154.1
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126328808105-19048-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:12:21.859066 20791 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:12:21.860553 20791 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:12:21.874688 20797 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:12:21.874977 20796 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:12:21.874993 20799 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:12:21.875656 20791 server_base.cc:1034] running on GCE node
I20260807 18:12:21.876317 20791 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:12:21.877353 20791 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:12:21.878580 20791 hybrid_clock.cc:648] HybridClock initialized: now 1786126341878505 us; error 51 us; skew 500 ppm
I20260807 18:12:21.878921 20791 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:12:21 dist-test-slave-gkw7 krb5kdc[20611](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126341, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:12:21.882642 20791 init.cc:377] Logged in from keytab as kudu/127.18.154.1@KRBTEST.COM (short username kudu)
I20260807 18:12:21.884232 20791 webserver.cc:460] Webserver started at http://127.18.154.1:45577/ using document root <none> and password file <none>
I20260807 18:12:21.885090 20791 fs_manager.cc:359] Metadata directory not provided
I20260807 18:12:21.885223 20791 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:12:21.885735 20791 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:12:21.889076 20791 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126328808105-19048-0/minicluster-data/ts-0/data/instance:
uuid: "4be4e3b09afc4a4d821fad7dc6b3bd87"
format_stamp: "Formatted at 2026-08-07 18:12:21 on dist-test-slave-gkw7"
server_key: "cf90563e72a274e66a068d044b590c8e"
server_key_iv: "e66b8bd2013b6770bbfb23c413d47269"
server_key_version: "encryptionkey@0"
I20260807 18:12:21.889987 20791 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126328808105-19048-0/minicluster-data/ts-0/wal/instance:
uuid: "4be4e3b09afc4a4d821fad7dc6b3bd87"
format_stamp: "Formatted at 2026-08-07 18:12:21 on dist-test-slave-gkw7"
server_key: "cf90563e72a274e66a068d044b590c8e"
server_key_iv: "e66b8bd2013b6770bbfb23c413d47269"
server_key_version: "encryptionkey@0"
I20260807 18:12:21.895815 20791 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.005s
I20260807 18:12:21.899562 20808 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:12:21.901036 20791 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 18:12:21.901222 20791 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126328808105-19048-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126328808105-19048-0/minicluster-data/ts-0/wal
uuid: "4be4e3b09afc4a4d821fad7dc6b3bd87"
format_stamp: "Formatted at 2026-08-07 18:12:21 on dist-test-slave-gkw7"
server_key: "cf90563e72a274e66a068d044b590c8e"
server_key_iv: "e66b8bd2013b6770bbfb23c413d47269"
server_key_version: "encryptionkey@0"
I20260807 18:12:21.901453 20791 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126328808105-19048-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126328808105-19048-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126328808105-19048-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:12:21.942148 20791 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:12:21.947065 20791 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:12:21.947378 20791 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:12:21.950536 20791 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:12:21.950660 20791 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:12:21.950863 20791 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:12:21.950955 20791 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 18:12:21 dist-test-slave-gkw7 krb5kdc[20611](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126341, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.1@KRBTEST.COM for kudu/127.18.154.62@KRBTEST.COM
I20260807 18:12:21.979501 20820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:21.951915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.1:50959 (local address 127.18.154.62:40323)
0807 18:12:21.952462 (+ 547us) server_negotiation.cc:206] Beginning negotiation
0807 18:12:21.952471 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:12:21.953320 (+ 849us) server_negotiation.cc:407] Connection header received
0807 18:12:21.956011 (+ 2691us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:12:21.956019 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:12:21.956139 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:12:21.956254 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:12:21.959350 (+ 3096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:21.960186 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:21.963666 (+ 3480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:21.964379 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:21.964549 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:21.970455 (+ 5906us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:12:21.970466 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:12:21.970474 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:12:21.970538 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:12:21.974857 (+ 4319us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:12:21.976185 (+ 1328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:12:21.976190 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:12:21.976192 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:12:21.976354 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:12:21.976937 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:12:21.976946 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:12:21.976953 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:12:21.977457 (+ 504us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:12:21.977696 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0807 18:12:21.978570 (+ 874us) server_negotiation.cc:299] Negotiation successful
0807 18:12:21.978923 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":402,"thread_start_us":285,"threads_started":1}
I20260807 18:12:21.980363 20821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:21.951983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:40323 (local address 127.18.154.1:50959)
0807 18:12:21.953117 (+ 1134us) negotiation.cc:107] Waiting for socket to connect
0807 18:12:21.953176 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0807 18:12:21.955579 (+ 2403us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:12:21.956932 (+ 1353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:12:21.956948 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:12:21.957993 (+ 1045us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:12:21.959101 (+ 1108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:12:21.959140 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:12:21.960362 (+ 1222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:12:21.960370 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:12:21.963423 (+ 3053us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:12:21.963451 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:12:21.964880 (+ 1429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:12:21.964885 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:12:21.965117 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:21.966024 (+ 907us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:12:21.966062 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:12:21.970071 (+ 4009us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:12:21.975069 (+ 4998us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:12:21.975143 (+ 74us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:12:21.975168 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:12:21.975978 (+ 810us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:12:21.976562 (+ 584us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:12:21.976565 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:12:21.976568 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:12:21.976776 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:12:21.977666 (+ 890us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:12:21.977673 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:12:21.978079 (+ 406us) client_negotiation.cc:769] Sending connection context
0807 18:12:21.978357 (+ 278us) client_negotiation.cc:240] Negotiation successful
0807 18:12:21.978949 (+ 592us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":779,"thread_start_us":471,"threads_started":1}
I20260807 18:12:22.010351 20820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:21.993295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.1:43957 (local address 127.18.154.62:40323)
0807 18:12:21.993560 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0807 18:12:21.993570 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:12:21.994189 (+ 619us) server_negotiation.cc:407] Connection header received
0807 18:12:21.994376 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:12:21.994386 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:12:21.994502 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:12:21.994659 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:12:21.995790 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:21.997101 (+ 1311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:21.999569 (+ 2468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:22.000305 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:22.000471 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:22.003562 (+ 3091us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:12:22.003568 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:12:22.003572 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:12:22.003610 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:12:22.007271 (+ 3661us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:12:22.008090 (+ 819us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:12:22.008094 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:12:22.008097 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:12:22.008238 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:12:22.008671 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:12:22.008681 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:12:22.008684 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:12:22.008980 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:12:22.009218 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0807 18:12:22.009677 (+ 459us) server_negotiation.cc:299] Negotiation successful
0807 18:12:22.009939 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260807 18:12:22.010659 20932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:21.993290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:40323 (local address 127.18.154.1:43957)
0807 18:12:21.994050 (+ 760us) negotiation.cc:107] Waiting for socket to connect
0807 18:12:21.994073 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 18:12:21.994235 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:12:21.994691 (+ 456us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:12:21.994695 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:12:21.995253 (+ 558us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:12:21.995588 (+ 335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:12:21.995600 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:12:21.997269 (+ 1669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:12:21.997282 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:12:21.999268 (+ 1986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:12:21.999290 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:12:22.000807 (+ 1517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:12:22.000817 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:12:22.001070 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:22.002284 (+ 1214us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:12:22.002308 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:12:22.003298 (+ 990us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:12:22.007441 (+ 4143us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:12:22.007447 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:12:22.007452 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:12:22.007926 (+ 474us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:12:22.008371 (+ 445us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:12:22.008377 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:12:22.008381 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:12:22.008534 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:12:22.009125 (+ 591us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:12:22.009134 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:12:22.009356 (+ 222us) client_negotiation.cc:769] Sending connection context
0807 18:12:22.009510 (+ 154us) client_negotiation.cc:240] Negotiation successful
0807 18:12:22.010070 (+ 560us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":585,"thread_start_us":418,"threads_started":1}
I20260807 18:12:22.017633 20791 rpc_server.cc:307] RPC server started. Bound to: 127.18.154.1:34831
I20260807 18:12:22.017653 20986 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.154.1:34831 every 8 connection(s)
I20260807 18:12:22.018893 20791 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126328808105-19048-0/minicluster-data/ts-0/data/info.pb
I20260807 18:12:22.021150 20988 heartbeater.cc:347] Connected to a master server at 127.18.154.62:40323
I20260807 18:12:22.021353 20988 heartbeater.cc:464] Registering TS with master...
I20260807 18:12:22.022104 20988 heartbeater.cc:511] Master 127.18.154.62:40323 requested a full tablet report, sending...
I20260807 18:12:22.024158 20681 ts_manager.cc:194] Registered new tserver with Master: 4be4e3b09afc4a4d821fad7dc6b3bd87 (127.18.154.1:34831)
I20260807 18:12:22.025705 20681 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.154.1@KRBTEST.COM'} at 127.18.154.1:50959
I20260807 18:12:22.027783 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 20791
I20260807 18:12:22.028007 19048 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126328808105-19048-0/minicluster-data/ts-0/wal/instance
I20260807 18:12:22.028692 19048 external_mini_cluster.cc:1428] Setting key e5ba7c1458885ecc402ca72e617326a4
WARNING: no policy specified for kudu/127.18.154.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.154.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.154.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.2.keytab.
Entry for principal kudu/127.18.154.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.2.keytab.
I20260807 18:12:22.077750 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126328808105-19048-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126328808105-19048-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126328808105-19048-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126328808105-19048-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.154.2:0
--local_ip_for_outbound_sockets=127.18.154.2
--webserver_interface=127.18.154.2
--webserver_port=0
--tserver_master_addrs=127.18.154.62:40323
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.2.keytab
--principal=kudu/127.18.154.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.154.20:42303
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126328808105-19048-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126328808105-19048-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126328808105-19048-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:12:22.418678 20993 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:12:22.419006 20993 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:12:22.419185 20993 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:12:22.426807 20993 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:12:22.426939 20993 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:12:22.427026 20993 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 18:12:22.427109 20993 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.154.2
I20260807 18:12:22.437312 20993 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.154.20:42303
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126328808105-19048-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126328808105-19048-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.2.keytab
--principal=kudu/127.18.154.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.154.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126328808105-19048-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.154.2
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.18.154.62:40323
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.154.2
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126328808105-19048-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:12:22.438750 20993 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:12:22.440095 20993 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:12:22.454404 20998 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:12:22.454432 21001 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:12:22.454581 21003 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:12:22.455535 20993 server_base.cc:1034] running on GCE node
I20260807 18:12:22.456200 20993 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:12:22.457086 20993 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:12:22.458323 20993 hybrid_clock.cc:648] HybridClock initialized: now 1786126342458191 us; error 90 us; skew 500 ppm
I20260807 18:12:22.458671 20993 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:12:22 dist-test-slave-gkw7 krb5kdc[20611](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126342, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:12:22.465758 20993 init.cc:377] Logged in from keytab as kudu/127.18.154.2@KRBTEST.COM (short username kudu)
I20260807 18:12:22.467454 20993 webserver.cc:460] Webserver started at http://127.18.154.2:42717/ using document root <none> and password file <none>
I20260807 18:12:22.468282 20993 fs_manager.cc:359] Metadata directory not provided
I20260807 18:12:22.468430 20993 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:12:22.468806 20993 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:12:22.472220 20993 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126328808105-19048-0/minicluster-data/ts-1/data/instance:
uuid: "36c9b6beec1645a1b0a94a0850f35ad0"
format_stamp: "Formatted at 2026-08-07 18:12:22 on dist-test-slave-gkw7"
server_key: "28e17a968e37591a65de7f2eb9292d4c"
server_key_iv: "c27aeb4b626c0cacde86cfee32014173"
server_key_version: "encryptionkey@0"
I20260807 18:12:22.473135 20993 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126328808105-19048-0/minicluster-data/ts-1/wal/instance:
uuid: "36c9b6beec1645a1b0a94a0850f35ad0"
format_stamp: "Formatted at 2026-08-07 18:12:22 on dist-test-slave-gkw7"
server_key: "28e17a968e37591a65de7f2eb9292d4c"
server_key_iv: "c27aeb4b626c0cacde86cfee32014173"
server_key_version: "encryptionkey@0"
I20260807 18:12:22.479019 20993 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260807 18:12:22.483103 21015 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:12:22.484663 20993 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.003s
I20260807 18:12:22.484879 20993 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126328808105-19048-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126328808105-19048-0/minicluster-data/ts-1/wal
uuid: "36c9b6beec1645a1b0a94a0850f35ad0"
format_stamp: "Formatted at 2026-08-07 18:12:22 on dist-test-slave-gkw7"
server_key: "28e17a968e37591a65de7f2eb9292d4c"
server_key_iv: "c27aeb4b626c0cacde86cfee32014173"
server_key_version: "encryptionkey@0"
I20260807 18:12:22.485174 20993 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126328808105-19048-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126328808105-19048-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126328808105-19048-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:12:22.515007 20993 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:12:22.519693 20993 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:12:22.520005 20993 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:12:22.523483 20993 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:12:22.523602 20993 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:12:22.523782 20993 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:12:22.523865 20993 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 18:12:22 dist-test-slave-gkw7 krb5kdc[20611](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126342, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.2@KRBTEST.COM for kudu/127.18.154.62@KRBTEST.COM
I20260807 18:12:22.548465 21030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:22.524247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:40323 (local address 127.18.154.2:49175)
0807 18:12:22.525239 (+ 992us) negotiation.cc:107] Waiting for socket to connect
0807 18:12:22.525306 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0807 18:12:22.527453 (+ 2147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:12:22.528353 (+ 900us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:12:22.528365 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:12:22.529196 (+ 831us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:12:22.530203 (+ 1007us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:12:22.530249 (+ 46us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:12:22.531787 (+ 1538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:12:22.531791 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:12:22.534213 (+ 2422us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:12:22.534230 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:12:22.535084 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:12:22.535093 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:12:22.535307 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:22.536126 (+ 819us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:12:22.536177 (+ 51us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:12:22.539314 (+ 3137us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:12:22.542731 (+ 3417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:12:22.542746 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:12:22.542766 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:12:22.543452 (+ 686us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:12:22.543949 (+ 497us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:12:22.543960 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:12:22.543964 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:12:22.544260 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:12:22.544984 (+ 724us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:12:22.544993 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:12:22.545650 (+ 657us) client_negotiation.cc:769] Sending connection context
0807 18:12:22.546003 (+ 353us) client_negotiation.cc:240] Negotiation successful
0807 18:12:22.546632 (+ 629us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":672,"thread_start_us":403,"threads_started":1}
I20260807 18:12:22.549256 21026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:22.523570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.2:49175 (local address 127.18.154.62:40323)
0807 18:12:22.524200 (+ 630us) server_negotiation.cc:206] Beginning negotiation
0807 18:12:22.524210 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:12:22.525446 (+ 1236us) server_negotiation.cc:407] Connection header received
0807 18:12:22.527750 (+ 2304us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:12:22.527759 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:12:22.527859 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:12:22.528012 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:12:22.530482 (+ 2470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:22.531613 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:22.534371 (+ 2758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:22.534817 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:22.534958 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:22.539554 (+ 4596us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:12:22.539559 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:12:22.539562 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:12:22.539597 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:12:22.542520 (+ 2923us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:12:22.543601 (+ 1081us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:12:22.543607 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:12:22.543611 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:12:22.543787 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:12:22.544397 (+ 610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:12:22.544404 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:12:22.544407 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:12:22.544846 (+ 439us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:12:22.545021 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0807 18:12:22.548581 (+ 3560us) server_negotiation.cc:299] Negotiation successful
0807 18:12:22.548875 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":464,"thread_start_us":296,"threads_started":1}
I20260807 18:12:22.573941 21026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:22.557276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.2:44307 (local address 127.18.154.62:40323)
0807 18:12:22.557497 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0807 18:12:22.557506 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:12:22.558254 (+ 748us) server_negotiation.cc:407] Connection header received
0807 18:12:22.558462 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:12:22.558466 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:12:22.558535 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:12:22.558622 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:12:22.560170 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:22.560890 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:22.563625 (+ 2735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:22.564316 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:22.564428 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:22.567565 (+ 3137us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:12:22.567574 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:12:22.567582 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:12:22.567627 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:12:22.571100 (+ 3473us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:12:22.571848 (+ 748us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:12:22.571853 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:12:22.571859 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:12:22.571988 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:12:22.572389 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:12:22.572393 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:12:22.572395 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:12:22.572728 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:12:22.572889 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0807 18:12:22.573311 (+ 422us) server_negotiation.cc:299] Negotiation successful
0807 18:12:22.573569 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260807 18:12:22.574121 21146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:22.556886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:40323 (local address 127.18.154.2:44307)
0807 18:12:22.558108 (+ 1222us) negotiation.cc:107] Waiting for socket to connect
0807 18:12:22.558135 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0807 18:12:22.558330 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:12:22.558702 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:12:22.558710 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:12:22.559481 (+ 771us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:12:22.559978 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:12:22.559996 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:12:22.561043 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:12:22.561050 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:12:22.563381 (+ 2331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:12:22.563406 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:12:22.564790 (+ 1384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:12:22.564799 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:12:22.565022 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:22.565897 (+ 875us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:12:22.565925 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:12:22.567299 (+ 1374us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:12:22.571250 (+ 3951us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:12:22.571257 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:12:22.571261 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:12:22.571705 (+ 444us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:12:22.572110 (+ 405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:12:22.572115 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:12:22.572119 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:12:22.572259 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:12:22.572853 (+ 594us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:12:22.572858 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:12:22.573051 (+ 193us) client_negotiation.cc:769] Sending connection context
0807 18:12:22.573193 (+ 142us) client_negotiation.cc:240] Negotiation successful
0807 18:12:22.573587 (+ 394us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1004,"thread_start_us":828,"threads_started":1}
I20260807 18:12:22.577368 20993 rpc_server.cc:307] RPC server started. Bound to: 127.18.154.2:40867
I20260807 18:12:22.577409 21196 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.154.2:40867 every 8 connection(s)
I20260807 18:12:22.578760 20993 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126328808105-19048-0/minicluster-data/ts-1/data/info.pb
I20260807 18:12:22.580823 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 20993
I20260807 18:12:22.581017 19048 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126328808105-19048-0/minicluster-data/ts-1/wal/instance
I20260807 18:12:22.581658 19048 external_mini_cluster.cc:1428] Setting key 02cb50bca41d73304ff4550493030766
I20260807 18:12:22.582185 21198 heartbeater.cc:347] Connected to a master server at 127.18.154.62:40323
I20260807 18:12:22.582458 21198 heartbeater.cc:464] Registering TS with master...
I20260807 18:12:22.583338 21198 heartbeater.cc:511] Master 127.18.154.62:40323 requested a full tablet report, sending...
I20260807 18:12:22.584967 20681 ts_manager.cc:194] Registered new tserver with Master: 36c9b6beec1645a1b0a94a0850f35ad0 (127.18.154.2:40867)
I20260807 18:12:22.585894 20681 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.154.2@KRBTEST.COM'} at 127.18.154.2:49175
WARNING: no policy specified for kudu/127.18.154.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.154.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.154.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.3.keytab.
Entry for principal kudu/127.18.154.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.3.keytab.
I20260807 18:12:22.629175 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126328808105-19048-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126328808105-19048-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126328808105-19048-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126328808105-19048-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.154.3:0
--local_ip_for_outbound_sockets=127.18.154.3
--webserver_interface=127.18.154.3
--webserver_port=0
--tserver_master_addrs=127.18.154.62:40323
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.3.keytab
--principal=kudu/127.18.154.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.154.20:42303
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126328808105-19048-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126328808105-19048-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126328808105-19048-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:12:22.965225 21203 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:12:22.965523 21203 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:12:22.965754 21203 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:12:22.973104 21203 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:12:22.973274 21203 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:12:22.973438 21203 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 18:12:22.973598 21203 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.154.3
I20260807 18:12:22.983906 21203 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.154.20:42303
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126328808105-19048-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126328808105-19048-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.3.keytab
--principal=kudu/127.18.154.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.154.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126328808105-19048-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.154.3
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.18.154.62:40323
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.154.3
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126328808105-19048-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:12:22.985334 21203 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:12:22.986732 21203 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:12:23.000491 21209 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:12:23.000782 21210 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:12:23.001134 21213 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:12:23.001645 21203 server_base.cc:1034] running on GCE node
I20260807 18:12:23.002326 21203 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:12:23.003343 21203 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:12:23.004598 21203 hybrid_clock.cc:648] HybridClock initialized: now 1786126343004533 us; error 41 us; skew 500 ppm
I20260807 18:12:23.004989 21203 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:12:23 dist-test-slave-gkw7 krb5kdc[20611](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126343, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:12:23.008435 21203 init.cc:377] Logged in from keytab as kudu/127.18.154.3@KRBTEST.COM (short username kudu)
I20260807 18:12:23.010298 21203 webserver.cc:460] Webserver started at http://127.18.154.3:43275/ using document root <none> and password file <none>
I20260807 18:12:23.011068 21203 fs_manager.cc:359] Metadata directory not provided
I20260807 18:12:23.011215 21203 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:12:23.011574 21203 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:12:23.014712 21203 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126328808105-19048-0/minicluster-data/ts-2/data/instance:
uuid: "99aa7ef775274a3db8e5f5704b9105a0"
format_stamp: "Formatted at 2026-08-07 18:12:23 on dist-test-slave-gkw7"
server_key: "289c4db4b1b5010fa9fcb74b3e8c215f"
server_key_iv: "859e06444f7317671a60c8fb7727ef17"
server_key_version: "encryptionkey@0"
I20260807 18:12:23.015630 21203 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126328808105-19048-0/minicluster-data/ts-2/wal/instance:
uuid: "99aa7ef775274a3db8e5f5704b9105a0"
format_stamp: "Formatted at 2026-08-07 18:12:23 on dist-test-slave-gkw7"
server_key: "289c4db4b1b5010fa9fcb74b3e8c215f"
server_key_iv: "859e06444f7317671a60c8fb7727ef17"
server_key_version: "encryptionkey@0"
I20260807 18:12:23.021314 21203 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260807 18:12:23.025089 21221 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:12:23.026641 21203 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:12:23.026818 21203 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126328808105-19048-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126328808105-19048-0/minicluster-data/ts-2/wal
uuid: "99aa7ef775274a3db8e5f5704b9105a0"
format_stamp: "Formatted at 2026-08-07 18:12:23 on dist-test-slave-gkw7"
server_key: "289c4db4b1b5010fa9fcb74b3e8c215f"
server_key_iv: "859e06444f7317671a60c8fb7727ef17"
server_key_version: "encryptionkey@0"
I20260807 18:12:23.027053 21203 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126328808105-19048-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126328808105-19048-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126328808105-19048-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:12:23.028592 20988 heartbeater.cc:503] Master 127.18.154.62:40323 was elected leader, sending a full tablet report...
I20260807 18:12:23.047873 21203 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:12:23.052634 21203 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:12:23.052925 21203 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:12:23.055907 21203 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:12:23.056025 21203 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:12:23.056243 21203 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:12:23.056360 21203 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
Aug 07 18:12:23 dist-test-slave-gkw7 krb5kdc[20611](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126343, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.3@KRBTEST.COM for kudu/127.18.154.62@KRBTEST.COM
I20260807 18:12:23.082465 21026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:23.056448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.3:55715 (local address 127.18.154.62:40323)
0807 18:12:23.056708 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0807 18:12:23.056718 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:12:23.058301 (+ 1583us) server_negotiation.cc:407] Connection header received
0807 18:12:23.061477 (+ 3176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:12:23.061483 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:12:23.061610 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:12:23.061718 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:12:23.064936 (+ 3218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:23.066018 (+ 1082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:23.068481 (+ 2463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:23.069131 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:23.069292 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:23.074869 (+ 5577us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:12:23.074875 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:12:23.074878 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:12:23.074917 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:12:23.078690 (+ 3773us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:12:23.079702 (+ 1012us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:12:23.079708 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:12:23.079712 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:12:23.079867 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:12:23.080397 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:12:23.080403 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:12:23.080407 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:12:23.080712 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:12:23.080870 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0807 18:12:23.081802 (+ 932us) server_negotiation.cc:299] Negotiation successful
0807 18:12:23.082039 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260807 18:12:23.083806 21234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:23.057144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:40323 (local address 127.18.154.3:55715)
0807 18:12:23.058103 (+ 959us) negotiation.cc:107] Waiting for socket to connect
0807 18:12:23.058158 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0807 18:12:23.061066 (+ 2908us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:12:23.062450 (+ 1384us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:12:23.062465 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:12:23.063545 (+ 1080us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:12:23.064696 (+ 1151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:12:23.064732 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:12:23.066194 (+ 1462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:12:23.066200 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:12:23.068301 (+ 2101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:12:23.068319 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:12:23.069411 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:12:23.069423 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:12:23.069846 (+ 423us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:23.070880 (+ 1034us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:12:23.070916 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:12:23.074552 (+ 3636us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:12:23.078881 (+ 4329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:12:23.078892 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:12:23.078913 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:12:23.079550 (+ 637us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:12:23.080002 (+ 452us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:12:23.080007 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:12:23.080010 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:12:23.080279 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:12:23.080861 (+ 582us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:12:23.080869 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:12:23.081270 (+ 401us) client_negotiation.cc:769] Sending connection context
0807 18:12:23.081649 (+ 379us) client_negotiation.cc:240] Negotiation successful
0807 18:12:23.082367 (+ 718us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":659,"thread_start_us":385,"threads_started":1}
I20260807 18:12:23.108526 21389 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.154.3:39957 every 8 connection(s)
I20260807 18:12:23.108558 21203 rpc_server.cc:307] RPC server started. Bound to: 127.18.154.3:39957
I20260807 18:12:23.109794 21337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:23.091510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:40323 (local address 127.18.154.3:50709)
0807 18:12:23.092300 (+ 790us) negotiation.cc:107] Waiting for socket to connect
0807 18:12:23.092321 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 18:12:23.092460 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:12:23.092859 (+ 399us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:12:23.092866 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:12:23.093754 (+ 888us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:12:23.094405 (+ 651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:12:23.094426 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:12:23.096186 (+ 1760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:12:23.096193 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:12:23.098040 (+ 1847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:12:23.098054 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:12:23.099268 (+ 1214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:12:23.099275 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:12:23.099522 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:23.100458 (+ 936us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:12:23.100485 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:12:23.101910 (+ 1425us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:12:23.106797 (+ 4887us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:12:23.106801 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:12:23.106804 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:12:23.107279 (+ 475us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:12:23.107840 (+ 561us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:12:23.107846 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:12:23.107849 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:12:23.108054 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:12:23.108817 (+ 763us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:12:23.108820 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:12:23.108989 (+ 169us) client_negotiation.cc:769] Sending connection context
0807 18:12:23.109112 (+ 123us) client_negotiation.cc:240] Negotiation successful
0807 18:12:23.109395 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":639,"thread_start_us":492,"threads_started":1}
I20260807 18:12:23.109901 21026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:23.091434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.3:50709 (local address 127.18.154.62:40323)
0807 18:12:23.091664 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0807 18:12:23.091672 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:12:23.092432 (+ 760us) server_negotiation.cc:407] Connection header received
0807 18:12:23.092572 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:12:23.092580 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:12:23.092678 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:12:23.092800 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:12:23.094609 (+ 1809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:23.095980 (+ 1371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:23.098204 (+ 2224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:23.098964 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:23.099203 (+ 239us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:23.102170 (+ 2967us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:12:23.102189 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:12:23.102196 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:12:23.102254 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:12:23.106647 (+ 4393us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:12:23.107459 (+ 812us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:12:23.107464 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:12:23.107468 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:12:23.107670 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:12:23.108278 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:12:23.108284 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:12:23.108291 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:12:23.108700 (+ 409us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:12:23.108855 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0807 18:12:23.109271 (+ 416us) server_negotiation.cc:299] Negotiation successful
0807 18:12:23.109509 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260807 18:12:23.110340 21203 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126328808105-19048-0/minicluster-data/ts-2/data/info.pb
I20260807 18:12:23.111029 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 21203
I20260807 18:12:23.111224 19048 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126328808105-19048-0/minicluster-data/ts-2/wal/instance
I20260807 18:12:23.111963 19048 external_mini_cluster.cc:1428] Setting key 02b6679e9b9f2b2583d69d6114a60b75
I20260807 18:12:23.113585 21392 heartbeater.cc:347] Connected to a master server at 127.18.154.62:40323
I20260807 18:12:23.113890 21392 heartbeater.cc:464] Registering TS with master...
I20260807 18:12:23.114777 21392 heartbeater.cc:511] Master 127.18.154.62:40323 requested a full tablet report, sending...
I20260807 18:12:23.116243 20680 ts_manager.cc:194] Registered new tserver with Master: 99aa7ef775274a3db8e5f5704b9105a0 (127.18.154.3:39957)
I20260807 18:12:23.117146 20680 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.154.3@KRBTEST.COM'} at 127.18.154.3:55715
I20260807 18:12:23.130602 19048 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 18:12:23 dist-test-slave-gkw7 krb5kdc[20611](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126343, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 18:12:23 dist-test-slave-gkw7 krb5kdc[20611](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126343, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.18.154.62@KRBTEST.COM
I20260807 18:12:23.187834 21026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:23.172898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43094 (local address 127.18.154.62:40323)
0807 18:12:23.173213 (+ 315us) server_negotiation.cc:206] Beginning negotiation
0807 18:12:23.173221 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:12:23.173409 (+ 188us) server_negotiation.cc:407] Connection header received
0807 18:12:23.173608 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:12:23.173614 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:12:23.173693 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:12:23.173783 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:12:23.175159 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:23.175909 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:23.177184 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:23.177688 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:23.177787 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:23.181963 (+ 4176us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:12:23.181968 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:12:23.181971 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:12:23.182006 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:12:23.184923 (+ 2917us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:12:23.185632 (+ 709us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:12:23.185637 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:12:23.185639 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:12:23.185768 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:12:23.186204 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:12:23.186210 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:12:23.186215 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:12:23.186523 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:12:23.186817 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0807 18:12:23.187181 (+ 364us) server_negotiation.cc:299] Negotiation successful
0807 18:12:23.187420 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
I20260807 18:12:23.192863 20680 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43094:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 18:12:23.199909 20680 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:12:23.237782 21418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:23.214904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39402 (local address 127.18.154.1:34831)
0807 18:12:23.228651 (+ 13747us) server_negotiation.cc:206] Beginning negotiation
0807 18:12:23.228664 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 18:12:23.228705 (+ 41us) server_negotiation.cc:407] Connection header received
0807 18:12:23.228844 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:12:23.228857 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:12:23.229227 (+ 370us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:12:23.229500 (+ 273us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:12:23.230862 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:23.232632 (+ 1770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:23.235340 (+ 2708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:23.236395 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:23.236576 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:23.236845 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0807 18:12:23.237196 (+ 351us) server_negotiation.cc:299] Negotiation successful
0807 18:12:23.237388 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13561,"thread_start_us":13415,"threads_started":1}
I20260807 18:12:23.237799 21408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:23.214068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.1:34831 (local address 127.0.0.1:39402)
0807 18:12:23.228284 (+ 14216us) negotiation.cc:107] Waiting for socket to connect
0807 18:12:23.228318 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0807 18:12:23.228487 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:12:23.229571 (+ 1084us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:12:23.229584 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:12:23.229633 (+ 49us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:12:23.230200 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:12:23.230215 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:12:23.232772 (+ 2557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:12:23.232780 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:12:23.235162 (+ 2382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:12:23.235180 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:12:23.236684 (+ 1504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:12:23.236697 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:12:23.236942 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:23.236983 (+ 41us) client_negotiation.cc:769] Sending connection context
0807 18:12:23.237128 (+ 145us) client_negotiation.cc:240] Negotiation successful
0807 18:12:23.237346 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14039,"thread_start_us":12014,"threads_started":1}
I20260807 18:12:23.240020 21409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:23.213178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.2:40867 (local address 127.0.0.1:50516)
0807 18:12:23.225986 (+ 12808us) negotiation.cc:107] Waiting for socket to connect
0807 18:12:23.226067 (+ 81us) client_negotiation.cc:174] Beginning negotiation
0807 18:12:23.226309 (+ 242us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:12:23.231404 (+ 5095us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:12:23.231408 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:12:23.231450 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:12:23.232209 (+ 759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:12:23.232224 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:12:23.234165 (+ 1941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:12:23.234173 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:12:23.236774 (+ 2601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:12:23.236790 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:12:23.239041 (+ 2251us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:12:23.239047 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:12:23.239258 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:23.239299 (+ 41us) client_negotiation.cc:769] Sending connection context
0807 18:12:23.239387 (+ 88us) client_negotiation.cc:240] Negotiation successful
0807 18:12:23.239531 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12444,"mutex_wait_us":137,"spinlock_wait_cycles":14464,"thread_start_us":14741,"threads_started":1}
I20260807 18:12:23.240059 21410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:23.213402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50516 (local address 127.18.154.2:40867)
0807 18:12:23.230751 (+ 17349us) server_negotiation.cc:206] Beginning negotiation
0807 18:12:23.230761 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:12:23.230789 (+ 28us) server_negotiation.cc:407] Connection header received
0807 18:12:23.230891 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:12:23.230897 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:12:23.231134 (+ 237us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:12:23.231571 (+ 437us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:12:23.232386 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:23.233967 (+ 1581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:23.236968 (+ 3001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:23.238482 (+ 1514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:23.238641 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:23.238855 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0807 18:12:23.239518 (+ 663us) server_negotiation.cc:299] Negotiation successful
0807 18:12:23.239720 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17182,"thread_start_us":17693,"threads_started":1}
I20260807 18:12:23.240573 21413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:23.213371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.3:39957 (local address 127.0.0.1:53334)
0807 18:12:23.226427 (+ 13056us) negotiation.cc:107] Waiting for socket to connect
0807 18:12:23.226451 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 18:12:23.226617 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:12:23.231397 (+ 4780us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:12:23.231402 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:12:23.231446 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:12:23.232079 (+ 633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:12:23.232090 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:12:23.234291 (+ 2201us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:12:23.234298 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:12:23.236774 (+ 2476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:12:23.236790 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:12:23.239158 (+ 2368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:12:23.239164 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:12:23.239327 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:23.239371 (+ 44us) client_negotiation.cc:769] Sending connection context
0807 18:12:23.239444 (+ 73us) client_negotiation.cc:240] Negotiation successful
0807 18:12:23.239547 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12897,"mutex_wait_us":126,"spinlock_wait_cycles":6400,"thread_start_us":12083,"threads_started":1}
I20260807 18:12:23.240829 21416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:23.214133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53334 (local address 127.18.154.3:39957)
0807 18:12:23.230406 (+ 16273us) server_negotiation.cc:206] Beginning negotiation
0807 18:12:23.230415 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:12:23.230452 (+ 37us) server_negotiation.cc:407] Connection header received
0807 18:12:23.230570 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:12:23.230577 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:12:23.230832 (+ 255us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:12:23.231025 (+ 193us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:12:23.232276 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:23.234133 (+ 1857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:23.236969 (+ 2836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:23.238549 (+ 1580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:23.238672 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:23.238882 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0807 18:12:23.239792 (+ 910us) server_negotiation.cc:299] Negotiation successful
0807 18:12:23.239970 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16065,"thread_start_us":15839,"threads_started":1}
I20260807 18:12:23.242897 20881 tablet_service.cc:1467] Processing CreateTablet for tablet 63f39289a0524e3c814ff266684ffd82 (DEFAULT_TABLE table=test-table [id=145678dd8f904991953c7967fc49e8b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:12:23.243963 21098 tablet_service.cc:1467] Processing CreateTablet for tablet 63f39289a0524e3c814ff266684ffd82 (DEFAULT_TABLE table=test-table [id=145678dd8f904991953c7967fc49e8b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:12:23.244004 21294 tablet_service.cc:1467] Processing CreateTablet for tablet 63f39289a0524e3c814ff266684ffd82 (DEFAULT_TABLE table=test-table [id=145678dd8f904991953c7967fc49e8b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:12:23.245111 20881 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 63f39289a0524e3c814ff266684ffd82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:12:23.245338 21294 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 63f39289a0524e3c814ff266684ffd82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:12:23.245404 21098 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 63f39289a0524e3c814ff266684ffd82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:12:23.257907 21419 tablet_bootstrap.cc:492] T 63f39289a0524e3c814ff266684ffd82 P 99aa7ef775274a3db8e5f5704b9105a0: Bootstrap starting.
I20260807 18:12:23.261238 21419 tablet_bootstrap.cc:654] T 63f39289a0524e3c814ff266684ffd82 P 99aa7ef775274a3db8e5f5704b9105a0: Neither blocks nor log segments found. Creating new log.
I20260807 18:12:23.261270 21423 tablet_bootstrap.cc:492] T 63f39289a0524e3c814ff266684ffd82 P 36c9b6beec1645a1b0a94a0850f35ad0: Bootstrap starting.
I20260807 18:12:23.262816 21419 log.cc:824] T 63f39289a0524e3c814ff266684ffd82 P 99aa7ef775274a3db8e5f5704b9105a0: Log is configured to *not* fsync() on all Append() calls
I20260807 18:12:23.263200 21424 tablet_bootstrap.cc:492] T 63f39289a0524e3c814ff266684ffd82 P 4be4e3b09afc4a4d821fad7dc6b3bd87: Bootstrap starting.
I20260807 18:12:23.264871 21423 tablet_bootstrap.cc:654] T 63f39289a0524e3c814ff266684ffd82 P 36c9b6beec1645a1b0a94a0850f35ad0: Neither blocks nor log segments found. Creating new log.
I20260807 18:12:23.266285 21423 log.cc:824] T 63f39289a0524e3c814ff266684ffd82 P 36c9b6beec1645a1b0a94a0850f35ad0: Log is configured to *not* fsync() on all Append() calls
I20260807 18:12:23.266701 21419 tablet_bootstrap.cc:492] T 63f39289a0524e3c814ff266684ffd82 P 99aa7ef775274a3db8e5f5704b9105a0: No bootstrap required, opened a new log
I20260807 18:12:23.267092 21419 ts_tablet_manager.cc:1397] T 63f39289a0524e3c814ff266684ffd82 P 99aa7ef775274a3db8e5f5704b9105a0: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260807 18:12:23.267469 21424 tablet_bootstrap.cc:654] T 63f39289a0524e3c814ff266684ffd82 P 4be4e3b09afc4a4d821fad7dc6b3bd87: Neither blocks nor log segments found. Creating new log.
I20260807 18:12:23.268719 21424 log.cc:824] T 63f39289a0524e3c814ff266684ffd82 P 4be4e3b09afc4a4d821fad7dc6b3bd87: Log is configured to *not* fsync() on all Append() calls
I20260807 18:12:23.273257 21424 tablet_bootstrap.cc:492] T 63f39289a0524e3c814ff266684ffd82 P 4be4e3b09afc4a4d821fad7dc6b3bd87: No bootstrap required, opened a new log
I20260807 18:12:23.273573 21423 tablet_bootstrap.cc:492] T 63f39289a0524e3c814ff266684ffd82 P 36c9b6beec1645a1b0a94a0850f35ad0: No bootstrap required, opened a new log
I20260807 18:12:23.273605 21424 ts_tablet_manager.cc:1397] T 63f39289a0524e3c814ff266684ffd82 P 4be4e3b09afc4a4d821fad7dc6b3bd87: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.004s
I20260807 18:12:23.274050 21423 ts_tablet_manager.cc:1397] T 63f39289a0524e3c814ff266684ffd82 P 36c9b6beec1645a1b0a94a0850f35ad0: Time spent bootstrapping tablet: real 0.013s user 0.009s sys 0.003s
I20260807 18:12:23.281322 21419 raft_consensus.cc:348] T 63f39289a0524e3c814ff266684ffd82 P 99aa7ef775274a3db8e5f5704b9105a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36c9b6beec1645a1b0a94a0850f35ad0" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 40867 } } peers { permanent_uuid: "99aa7ef775274a3db8e5f5704b9105a0" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 39957 } } peers { permanent_uuid: "4be4e3b09afc4a4d821fad7dc6b3bd87" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 34831 } }
I20260807 18:12:23.281921 21419 raft_consensus.cc:374] T 63f39289a0524e3c814ff266684ffd82 P 99aa7ef775274a3db8e5f5704b9105a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:12:23.282023 21419 raft_consensus.cc:729] T 63f39289a0524e3c814ff266684ffd82 P 99aa7ef775274a3db8e5f5704b9105a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99aa7ef775274a3db8e5f5704b9105a0, State: Initialized, Role: FOLLOWER
I20260807 18:12:23.282771 21419 consensus_queue.cc:260] T 63f39289a0524e3c814ff266684ffd82 P 99aa7ef775274a3db8e5f5704b9105a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36c9b6beec1645a1b0a94a0850f35ad0" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 40867 } } peers { permanent_uuid: "99aa7ef775274a3db8e5f5704b9105a0" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 39957 } } peers { permanent_uuid: "4be4e3b09afc4a4d821fad7dc6b3bd87" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 34831 } }
I20260807 18:12:23.284101 21392 heartbeater.cc:503] Master 127.18.154.62:40323 was elected leader, sending a full tablet report...
I20260807 18:12:23.284795 21419 ts_tablet_manager.cc:1428] T 63f39289a0524e3c814ff266684ffd82 P 99aa7ef775274a3db8e5f5704b9105a0: Time spent starting tablet: real 0.018s user 0.010s sys 0.004s
I20260807 18:12:23.284889 21424 raft_consensus.cc:348] T 63f39289a0524e3c814ff266684ffd82 P 4be4e3b09afc4a4d821fad7dc6b3bd87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36c9b6beec1645a1b0a94a0850f35ad0" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 40867 } } peers { permanent_uuid: "99aa7ef775274a3db8e5f5704b9105a0" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 39957 } } peers { permanent_uuid: "4be4e3b09afc4a4d821fad7dc6b3bd87" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 34831 } }
I20260807 18:12:23.285362 21424 raft_consensus.cc:374] T 63f39289a0524e3c814ff266684ffd82 P 4be4e3b09afc4a4d821fad7dc6b3bd87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:12:23.285495 21424 raft_consensus.cc:729] T 63f39289a0524e3c814ff266684ffd82 P 4be4e3b09afc4a4d821fad7dc6b3bd87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4be4e3b09afc4a4d821fad7dc6b3bd87, State: Initialized, Role: FOLLOWER
I20260807 18:12:23.286645 21424 consensus_queue.cc:260] T 63f39289a0524e3c814ff266684ffd82 P 4be4e3b09afc4a4d821fad7dc6b3bd87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36c9b6beec1645a1b0a94a0850f35ad0" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 40867 } } peers { permanent_uuid: "99aa7ef775274a3db8e5f5704b9105a0" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 39957 } } peers { permanent_uuid: "4be4e3b09afc4a4d821fad7dc6b3bd87" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 34831 } }
I20260807 18:12:23.287125 21423 raft_consensus.cc:348] T 63f39289a0524e3c814ff266684ffd82 P 36c9b6beec1645a1b0a94a0850f35ad0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36c9b6beec1645a1b0a94a0850f35ad0" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 40867 } } peers { permanent_uuid: "99aa7ef775274a3db8e5f5704b9105a0" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 39957 } } peers { permanent_uuid: "4be4e3b09afc4a4d821fad7dc6b3bd87" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 34831 } }
I20260807 18:12:23.287622 21423 raft_consensus.cc:374] T 63f39289a0524e3c814ff266684ffd82 P 36c9b6beec1645a1b0a94a0850f35ad0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:12:23.287734 21423 raft_consensus.cc:729] T 63f39289a0524e3c814ff266684ffd82 P 36c9b6beec1645a1b0a94a0850f35ad0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36c9b6beec1645a1b0a94a0850f35ad0, State: Initialized, Role: FOLLOWER
I20260807 18:12:23.288434 21424 ts_tablet_manager.cc:1428] T 63f39289a0524e3c814ff266684ffd82 P 4be4e3b09afc4a4d821fad7dc6b3bd87: Time spent starting tablet: real 0.015s user 0.011s sys 0.002s
I20260807 18:12:23.288556 21423 consensus_queue.cc:260] T 63f39289a0524e3c814ff266684ffd82 P 36c9b6beec1645a1b0a94a0850f35ad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36c9b6beec1645a1b0a94a0850f35ad0" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 40867 } } peers { permanent_uuid: "99aa7ef775274a3db8e5f5704b9105a0" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 39957 } } peers { permanent_uuid: "4be4e3b09afc4a4d821fad7dc6b3bd87" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 34831 } }
I20260807 18:12:23.289999 21198 heartbeater.cc:503] Master 127.18.154.62:40323 was elected leader, sending a full tablet report...
I20260807 18:12:23.290887 21423 ts_tablet_manager.cc:1428] T 63f39289a0524e3c814ff266684ffd82 P 36c9b6beec1645a1b0a94a0850f35ad0: Time spent starting tablet: real 0.017s user 0.014s sys 0.001s
W20260807 18:12:23.330993 21200 tablet.cc:2369] T 63f39289a0524e3c814ff266684ffd82 P 36c9b6beec1645a1b0a94a0850f35ad0: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 18:12:23.366910 21393 tablet.cc:2369] T 63f39289a0524e3c814ff266684ffd82 P 99aa7ef775274a3db8e5f5704b9105a0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:12:23.483269 21430 raft_consensus.cc:482] T 63f39289a0524e3c814ff266684ffd82 P 4be4e3b09afc4a4d821fad7dc6b3bd87 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:12:23.483649 21430 raft_consensus.cc:504] T 63f39289a0524e3c814ff266684ffd82 P 4be4e3b09afc4a4d821fad7dc6b3bd87 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36c9b6beec1645a1b0a94a0850f35ad0" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 40867 } } peers { permanent_uuid: "99aa7ef775274a3db8e5f5704b9105a0" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 39957 } } peers { permanent_uuid: "4be4e3b09afc4a4d821fad7dc6b3bd87" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 34831 } }
I20260807 18:12:23.485618 21430 leader_election.cc:288] T 63f39289a0524e3c814ff266684ffd82 P 4be4e3b09afc4a4d821fad7dc6b3bd87 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 36c9b6beec1645a1b0a94a0850f35ad0 (127.18.154.2:40867), 99aa7ef775274a3db8e5f5704b9105a0 (127.18.154.3:39957)
I20260807 18:12:23.494252 21435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:23.485884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.2:40867 (local address 127.18.154.1:59891)
0807 18:12:23.487411 (+ 1527us) negotiation.cc:107] Waiting for socket to connect
0807 18:12:23.487434 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 18:12:23.487716 (+ 282us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:12:23.488114 (+ 398us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:12:23.488119 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:12:23.488145 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:12:23.488753 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:12:23.488774 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:12:23.490175 (+ 1401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:12:23.490183 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:12:23.492007 (+ 1824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:12:23.492020 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:12:23.493308 (+ 1288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:12:23.493315 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:12:23.493510 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:23.493596 (+ 86us) client_negotiation.cc:769] Sending connection context
0807 18:12:23.493725 (+ 129us) client_negotiation.cc:240] Negotiation successful
0807 18:12:23.493868 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1303,"mutex_wait_us":23,"spinlock_wait_cycles":47616,"thread_start_us":1011,"threads_started":1}
I20260807 18:12:23.494279 21410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:23.486105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.1:59891 (local address 127.18.154.2:40867)
0807 18:12:23.486338 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0807 18:12:23.486345 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:12:23.487579 (+ 1234us) server_negotiation.cc:407] Connection header received
0807 18:12:23.487906 (+ 327us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:12:23.487910 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:12:23.487971 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:12:23.488086 (+ 115us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:12:23.488928 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:23.490042 (+ 1114us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:23.492184 (+ 2142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:23.493022 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:23.493161 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:23.493212 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0807 18:12:23.493816 (+ 604us) server_negotiation.cc:299] Negotiation successful
0807 18:12:23.493958 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260807 18:12:23.494635 21416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:23.486161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.1:45853 (local address 127.18.154.3:39957)
0807 18:12:23.486351 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0807 18:12:23.486358 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:12:23.487640 (+ 1282us) server_negotiation.cc:407] Connection header received
0807 18:12:23.487867 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:12:23.487879 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:12:23.487943 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:12:23.488014 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:12:23.489215 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:23.490143 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:23.492296 (+ 2153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:23.493643 (+ 1347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:23.493799 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:23.493873 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0807 18:12:23.494249 (+ 376us) server_negotiation.cc:299] Negotiation successful
0807 18:12:23.494407 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260807 18:12:23.494776 21432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:23.485986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.3:39957 (local address 127.18.154.1:45853)
0807 18:12:23.487482 (+ 1496us) negotiation.cc:107] Waiting for socket to connect
0807 18:12:23.487503 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 18:12:23.487722 (+ 219us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:12:23.488061 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:12:23.488068 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:12:23.488226 (+ 158us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:12:23.489022 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:12:23.489038 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:12:23.490319 (+ 1281us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:12:23.490323 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:12:23.492106 (+ 1783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:12:23.492123 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:12:23.493892 (+ 1769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:12:23.493899 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:12:23.494060 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:23.494098 (+ 38us) client_negotiation.cc:769] Sending connection context
0807 18:12:23.494201 (+ 103us) client_negotiation.cc:240] Negotiation successful
0807 18:12:23.494342 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1283,"mutex_wait_us":65,"spinlock_wait_cycles":59520,"thread_start_us":717,"threads_started":1}
I20260807 18:12:23.495651 21126 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "63f39289a0524e3c814ff266684ffd82" candidate_uuid: "4be4e3b09afc4a4d821fad7dc6b3bd87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36c9b6beec1645a1b0a94a0850f35ad0" is_pre_election: true
I20260807 18:12:23.495659 21327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "63f39289a0524e3c814ff266684ffd82" candidate_uuid: "4be4e3b09afc4a4d821fad7dc6b3bd87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99aa7ef775274a3db8e5f5704b9105a0" is_pre_election: true
I20260807 18:12:23.496181 21126 raft_consensus.cc:2454] T 63f39289a0524e3c814ff266684ffd82 P 36c9b6beec1645a1b0a94a0850f35ad0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4be4e3b09afc4a4d821fad7dc6b3bd87 in term 0.
I20260807 18:12:23.496181 21327 raft_consensus.cc:2454] T 63f39289a0524e3c814ff266684ffd82 P 99aa7ef775274a3db8e5f5704b9105a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4be4e3b09afc4a4d821fad7dc6b3bd87 in term 0.
I20260807 18:12:23.496827 20809 leader_election.cc:302] T 63f39289a0524e3c814ff266684ffd82 P 4be4e3b09afc4a4d821fad7dc6b3bd87 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4be4e3b09afc4a4d821fad7dc6b3bd87, 99aa7ef775274a3db8e5f5704b9105a0; no voters:
I20260807 18:12:23.497229 21430 raft_consensus.cc:2781] T 63f39289a0524e3c814ff266684ffd82 P 4be4e3b09afc4a4d821fad7dc6b3bd87 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:12:23.497511 21430 raft_consensus.cc:482] T 63f39289a0524e3c814ff266684ffd82 P 4be4e3b09afc4a4d821fad7dc6b3bd87 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:12:23.497735 21430 raft_consensus.cc:3037] T 63f39289a0524e3c814ff266684ffd82 P 4be4e3b09afc4a4d821fad7dc6b3bd87 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:12:23.500371 21430 raft_consensus.cc:504] T 63f39289a0524e3c814ff266684ffd82 P 4be4e3b09afc4a4d821fad7dc6b3bd87 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36c9b6beec1645a1b0a94a0850f35ad0" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 40867 } } peers { permanent_uuid: "99aa7ef775274a3db8e5f5704b9105a0" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 39957 } } peers { permanent_uuid: "4be4e3b09afc4a4d821fad7dc6b3bd87" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 34831 } }
I20260807 18:12:23.501001 21430 leader_election.cc:288] T 63f39289a0524e3c814ff266684ffd82 P 4be4e3b09afc4a4d821fad7dc6b3bd87 [CANDIDATE]: Term 1 election: Requested vote from peers 36c9b6beec1645a1b0a94a0850f35ad0 (127.18.154.2:40867), 99aa7ef775274a3db8e5f5704b9105a0 (127.18.154.3:39957)
I20260807 18:12:23.501634 21126 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "63f39289a0524e3c814ff266684ffd82" candidate_uuid: "4be4e3b09afc4a4d821fad7dc6b3bd87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36c9b6beec1645a1b0a94a0850f35ad0"
I20260807 18:12:23.501809 21327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "63f39289a0524e3c814ff266684ffd82" candidate_uuid: "4be4e3b09afc4a4d821fad7dc6b3bd87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99aa7ef775274a3db8e5f5704b9105a0"
I20260807 18:12:23.501977 21126 raft_consensus.cc:3037] T 63f39289a0524e3c814ff266684ffd82 P 36c9b6beec1645a1b0a94a0850f35ad0 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:12:23.502039 21327 raft_consensus.cc:3037] T 63f39289a0524e3c814ff266684ffd82 P 99aa7ef775274a3db8e5f5704b9105a0 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:12:23.505772 21327 raft_consensus.cc:2454] T 63f39289a0524e3c814ff266684ffd82 P 99aa7ef775274a3db8e5f5704b9105a0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4be4e3b09afc4a4d821fad7dc6b3bd87 in term 1.
I20260807 18:12:23.505790 21126 raft_consensus.cc:2454] T 63f39289a0524e3c814ff266684ffd82 P 36c9b6beec1645a1b0a94a0850f35ad0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4be4e3b09afc4a4d821fad7dc6b3bd87 in term 1.
I20260807 18:12:23.506335 20812 leader_election.cc:302] T 63f39289a0524e3c814ff266684ffd82 P 4be4e3b09afc4a4d821fad7dc6b3bd87 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 36c9b6beec1645a1b0a94a0850f35ad0, 4be4e3b09afc4a4d821fad7dc6b3bd87, 99aa7ef775274a3db8e5f5704b9105a0; no voters:
I20260807 18:12:23.506717 21430 raft_consensus.cc:2781] T 63f39289a0524e3c814ff266684ffd82 P 4be4e3b09afc4a4d821fad7dc6b3bd87 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:12:23.507228 21430 raft_consensus.cc:686] T 63f39289a0524e3c814ff266684ffd82 P 4be4e3b09afc4a4d821fad7dc6b3bd87 [term 1 LEADER]: Becoming Leader. State: Replica: 4be4e3b09afc4a4d821fad7dc6b3bd87, State: Running, Role: LEADER
I20260807 18:12:23.507752 21430 consensus_queue.cc:237] T 63f39289a0524e3c814ff266684ffd82 P 4be4e3b09afc4a4d821fad7dc6b3bd87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36c9b6beec1645a1b0a94a0850f35ad0" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 40867 } } peers { permanent_uuid: "99aa7ef775274a3db8e5f5704b9105a0" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 39957 } } peers { permanent_uuid: "4be4e3b09afc4a4d821fad7dc6b3bd87" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 34831 } }
I20260807 18:12:23.512832 20681 catalog_manager.cc:5495] T 63f39289a0524e3c814ff266684ffd82 P 4be4e3b09afc4a4d821fad7dc6b3bd87 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4be4e3b09afc4a4d821fad7dc6b3bd87 (127.18.154.1). New cstate: current_term: 1 leader_uuid: "4be4e3b09afc4a4d821fad7dc6b3bd87" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36c9b6beec1645a1b0a94a0850f35ad0" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 40867 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "99aa7ef775274a3db8e5f5704b9105a0" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 39957 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4be4e3b09afc4a4d821fad7dc6b3bd87" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 34831 } health_report { overall_health: HEALTHY } } }
W20260807 18:12:23.521682 20989 tablet.cc:2369] T 63f39289a0524e3c814ff266684ffd82 P 4be4e3b09afc4a4d821fad7dc6b3bd87: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:12:23.552126 21418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:23.545713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39412 (local address 127.18.154.1:34831)
0807 18:12:23.546010 (+ 297us) server_negotiation.cc:206] Beginning negotiation
0807 18:12:23.546018 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:12:23.546047 (+ 29us) server_negotiation.cc:407] Connection header received
0807 18:12:23.546434 (+ 387us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:12:23.546438 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:12:23.546528 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:12:23.546652 (+ 124us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:12:23.547059 (+ 407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:23.547826 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:23.549354 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:23.550049 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:23.550170 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:23.550720 (+ 550us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:12:23.551380 (+ 660us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:12:23.551615 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0807 18:12:23.551724 (+ 109us) server_negotiation.cc:299] Negotiation successful
0807 18:12:23.551806 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260807 18:12:23.561971 21126 raft_consensus.cc:1261] T 63f39289a0524e3c814ff266684ffd82 P 36c9b6beec1645a1b0a94a0850f35ad0 [term 1 FOLLOWER]: Refusing update from remote peer 4be4e3b09afc4a4d821fad7dc6b3bd87: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:12:23.563169 21436 consensus_queue.cc:1035] T 63f39289a0524e3c814ff266684ffd82 P 4be4e3b09afc4a4d821fad7dc6b3bd87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "36c9b6beec1645a1b0a94a0850f35ad0" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 40867 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:12:23.563493 21327 raft_consensus.cc:1261] T 63f39289a0524e3c814ff266684ffd82 P 99aa7ef775274a3db8e5f5704b9105a0 [term 1 FOLLOWER]: Refusing update from remote peer 4be4e3b09afc4a4d821fad7dc6b3bd87: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:12:23.564429 21436 consensus_queue.cc:1035] T 63f39289a0524e3c814ff266684ffd82 P 4be4e3b09afc4a4d821fad7dc6b3bd87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99aa7ef775274a3db8e5f5704b9105a0" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 39957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:12:23.577586 21441 mvcc.cc:205] Tried to move back new op lower bound from 7315973503217557504 to 7315973503011524608. Current Snapshot: MvccSnapshot[applied={T|T < 7315973503217557504}]
I20260807 18:12:23.606617 20680 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43094:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:12:23.607050 20680 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43094:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:12:23.611891 20680 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 98661d04c87d4fd0834cfb4532531d75: Sending DeleteTablet for 3 replicas of tablet 63f39289a0524e3c814ff266684ffd82
I20260807 18:12:23.613396 21098 tablet_service.cc:1514] Processing DeleteTablet for tablet 63f39289a0524e3c814ff266684ffd82 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:12:23 UTC) from {username='kudu', principal='kudu/127.18.154.62@KRBTEST.COM'} at 127.0.0.1:50516
I20260807 18:12:23.613375 21294 tablet_service.cc:1514] Processing DeleteTablet for tablet 63f39289a0524e3c814ff266684ffd82 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:12:23 UTC) from {username='kudu', principal='kudu/127.18.154.62@KRBTEST.COM'} at 127.0.0.1:53334
I20260807 18:12:23.613411 20881 tablet_service.cc:1514] Processing DeleteTablet for tablet 63f39289a0524e3c814ff266684ffd82 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:12:23 UTC) from {username='kudu', principal='kudu/127.18.154.62@KRBTEST.COM'} at 127.0.0.1:39402
I20260807 18:12:23.618880 20680 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43094:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 18:12:23.619773 20680 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:12:23.622260 21452 tablet_replica.cc:330] T 63f39289a0524e3c814ff266684ffd82 P 99aa7ef775274a3db8e5f5704b9105a0: stopping tablet replica
I20260807 18:12:23.622920 21451 tablet_replica.cc:330] T 63f39289a0524e3c814ff266684ffd82 P 4be4e3b09afc4a4d821fad7dc6b3bd87: stopping tablet replica
I20260807 18:12:23.622959 21452 raft_consensus.cc:2229] T 63f39289a0524e3c814ff266684ffd82 P 99aa7ef775274a3db8e5f5704b9105a0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:12:23.623378 21451 raft_consensus.cc:2229] T 63f39289a0524e3c814ff266684ffd82 P 4be4e3b09afc4a4d821fad7dc6b3bd87 [term 1 LEADER]: Raft consensus shutting down.
I20260807 18:12:23.623466 21452 raft_consensus.cc:2258] T 63f39289a0524e3c814ff266684ffd82 P 99aa7ef775274a3db8e5f5704b9105a0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:12:23.624238 21451 raft_consensus.cc:2258] T 63f39289a0524e3c814ff266684ffd82 P 4be4e3b09afc4a4d821fad7dc6b3bd87 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:12:23.626799 21452 ts_tablet_manager.cc:1905] T 63f39289a0524e3c814ff266684ffd82 P 99aa7ef775274a3db8e5f5704b9105a0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:12:23.631008 21451 ts_tablet_manager.cc:1905] T 63f39289a0524e3c814ff266684ffd82 P 4be4e3b09afc4a4d821fad7dc6b3bd87: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:12:23.631198 21294 tablet_service.cc:1467] Processing CreateTablet for tablet 33b7d5507e7a40b88cef2e6e33bdad1b (DEFAULT_TABLE table=test-table [id=aff22c9b6da44025a4364dcc8a5ab141]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:12:23.631446 21097 tablet_service.cc:1467] Processing CreateTablet for tablet 33b7d5507e7a40b88cef2e6e33bdad1b (DEFAULT_TABLE table=test-table [id=aff22c9b6da44025a4364dcc8a5ab141]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:12:23.631753 21294 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 33b7d5507e7a40b88cef2e6e33bdad1b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:12:23.632203 21097 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 33b7d5507e7a40b88cef2e6e33bdad1b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:12:23.638659 20881 tablet_service.cc:1467] Processing CreateTablet for tablet 33b7d5507e7a40b88cef2e6e33bdad1b (DEFAULT_TABLE table=test-table [id=aff22c9b6da44025a4364dcc8a5ab141]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:12:23.638988 21419 tablet_bootstrap.cc:492] T 33b7d5507e7a40b88cef2e6e33bdad1b P 99aa7ef775274a3db8e5f5704b9105a0: Bootstrap starting.
I20260807 18:12:23.639218 20881 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 33b7d5507e7a40b88cef2e6e33bdad1b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:12:23.640941 21419 tablet_bootstrap.cc:654] T 33b7d5507e7a40b88cef2e6e33bdad1b P 99aa7ef775274a3db8e5f5704b9105a0: Neither blocks nor log segments found. Creating new log.
I20260807 18:12:23.645488 21452 ts_tablet_manager.cc:1918] T 63f39289a0524e3c814ff266684ffd82 P 99aa7ef775274a3db8e5f5704b9105a0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:12:23.645850 21452 log.cc:1196] T 63f39289a0524e3c814ff266684ffd82 P 99aa7ef775274a3db8e5f5704b9105a0: Deleting WAL directory at /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126328808105-19048-0/minicluster-data/ts-2/wal/wals/63f39289a0524e3c814ff266684ffd82
I20260807 18:12:23.646499 21452 ts_tablet_manager.cc:1939] T 63f39289a0524e3c814ff266684ffd82 P 99aa7ef775274a3db8e5f5704b9105a0: Deleting consensus metadata
I20260807 18:12:23.648751 20656 catalog_manager.cc:4841] TS 99aa7ef775274a3db8e5f5704b9105a0 (127.18.154.3:39957): tablet 63f39289a0524e3c814ff266684ffd82 (table test-table [id=145678dd8f904991953c7967fc49e8b7]) successfully deleted
I20260807 18:12:23.651255 21423 tablet_bootstrap.cc:492] T 33b7d5507e7a40b88cef2e6e33bdad1b P 36c9b6beec1645a1b0a94a0850f35ad0: Bootstrap starting.
I20260807 18:12:23.653344 21451 ts_tablet_manager.cc:1918] T 63f39289a0524e3c814ff266684ffd82 P 4be4e3b09afc4a4d821fad7dc6b3bd87: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:12:23.653566 21423 tablet_bootstrap.cc:654] T 33b7d5507e7a40b88cef2e6e33bdad1b P 36c9b6beec1645a1b0a94a0850f35ad0: Neither blocks nor log segments found. Creating new log.
I20260807 18:12:23.653571 21451 log.cc:1196] T 63f39289a0524e3c814ff266684ffd82 P 4be4e3b09afc4a4d821fad7dc6b3bd87: Deleting WAL directory at /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126328808105-19048-0/minicluster-data/ts-0/wal/wals/63f39289a0524e3c814ff266684ffd82
I20260807 18:12:23.654225 21451 ts_tablet_manager.cc:1939] T 63f39289a0524e3c814ff266684ffd82 P 4be4e3b09afc4a4d821fad7dc6b3bd87: Deleting consensus metadata
I20260807 18:12:23.656272 20657 catalog_manager.cc:4841] TS 4be4e3b09afc4a4d821fad7dc6b3bd87 (127.18.154.1:34831): tablet 63f39289a0524e3c814ff266684ffd82 (table test-table [id=145678dd8f904991953c7967fc49e8b7]) successfully deleted
I20260807 18:12:23.659848 21419 tablet_bootstrap.cc:492] T 33b7d5507e7a40b88cef2e6e33bdad1b P 99aa7ef775274a3db8e5f5704b9105a0: No bootstrap required, opened a new log
I20260807 18:12:23.660022 21419 ts_tablet_manager.cc:1397] T 33b7d5507e7a40b88cef2e6e33bdad1b P 99aa7ef775274a3db8e5f5704b9105a0: Time spent bootstrapping tablet: real 0.021s user 0.010s sys 0.000s
I20260807 18:12:23.660977 21419 raft_consensus.cc:348] T 33b7d5507e7a40b88cef2e6e33bdad1b P 99aa7ef775274a3db8e5f5704b9105a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4be4e3b09afc4a4d821fad7dc6b3bd87" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 34831 } } peers { permanent_uuid: "36c9b6beec1645a1b0a94a0850f35ad0" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 40867 } } peers { permanent_uuid: "99aa7ef775274a3db8e5f5704b9105a0" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 39957 } }
I20260807 18:12:23.661266 21419 raft_consensus.cc:374] T 33b7d5507e7a40b88cef2e6e33bdad1b P 99aa7ef775274a3db8e5f5704b9105a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:12:23.661381 21419 raft_consensus.cc:729] T 33b7d5507e7a40b88cef2e6e33bdad1b P 99aa7ef775274a3db8e5f5704b9105a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99aa7ef775274a3db8e5f5704b9105a0, State: Initialized, Role: FOLLOWER
I20260807 18:12:23.661736 21419 consensus_queue.cc:260] T 33b7d5507e7a40b88cef2e6e33bdad1b P 99aa7ef775274a3db8e5f5704b9105a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4be4e3b09afc4a4d821fad7dc6b3bd87" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 34831 } } peers { permanent_uuid: "36c9b6beec1645a1b0a94a0850f35ad0" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 40867 } } peers { permanent_uuid: "99aa7ef775274a3db8e5f5704b9105a0" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 39957 } }
I20260807 18:12:23.662428 21419 ts_tablet_manager.cc:1428] T 33b7d5507e7a40b88cef2e6e33bdad1b P 99aa7ef775274a3db8e5f5704b9105a0: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 18:12:23.666185 21424 tablet_bootstrap.cc:492] T 33b7d5507e7a40b88cef2e6e33bdad1b P 4be4e3b09afc4a4d821fad7dc6b3bd87: Bootstrap starting.
I20260807 18:12:23.668278 21423 tablet_bootstrap.cc:492] T 33b7d5507e7a40b88cef2e6e33bdad1b P 36c9b6beec1645a1b0a94a0850f35ad0: No bootstrap required, opened a new log
I20260807 18:12:23.668467 21423 ts_tablet_manager.cc:1397] T 33b7d5507e7a40b88cef2e6e33bdad1b P 36c9b6beec1645a1b0a94a0850f35ad0: Time spent bootstrapping tablet: real 0.017s user 0.001s sys 0.007s
I20260807 18:12:23.668668 21424 tablet_bootstrap.cc:654] T 33b7d5507e7a40b88cef2e6e33bdad1b P 4be4e3b09afc4a4d821fad7dc6b3bd87: Neither blocks nor log segments found. Creating new log.
I20260807 18:12:23.669432 21423 raft_consensus.cc:348] T 33b7d5507e7a40b88cef2e6e33bdad1b P 36c9b6beec1645a1b0a94a0850f35ad0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4be4e3b09afc4a4d821fad7dc6b3bd87" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 34831 } } peers { permanent_uuid: "36c9b6beec1645a1b0a94a0850f35ad0" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 40867 } } peers { permanent_uuid: "99aa7ef775274a3db8e5f5704b9105a0" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 39957 } }
I20260807 18:12:23.669728 21423 raft_consensus.cc:374] T 33b7d5507e7a40b88cef2e6e33bdad1b P 36c9b6beec1645a1b0a94a0850f35ad0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:12:23.669847 21423 raft_consensus.cc:729] T 33b7d5507e7a40b88cef2e6e33bdad1b P 36c9b6beec1645a1b0a94a0850f35ad0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36c9b6beec1645a1b0a94a0850f35ad0, State: Initialized, Role: FOLLOWER
I20260807 18:12:23.670184 21423 consensus_queue.cc:260] T 33b7d5507e7a40b88cef2e6e33bdad1b P 36c9b6beec1645a1b0a94a0850f35ad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4be4e3b09afc4a4d821fad7dc6b3bd87" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 34831 } } peers { permanent_uuid: "36c9b6beec1645a1b0a94a0850f35ad0" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 40867 } } peers { permanent_uuid: "99aa7ef775274a3db8e5f5704b9105a0" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 39957 } }
I20260807 18:12:23.673379 21423 ts_tablet_manager.cc:1428] T 33b7d5507e7a40b88cef2e6e33bdad1b P 36c9b6beec1645a1b0a94a0850f35ad0: Time spent starting tablet: real 0.005s user 0.003s sys 0.000s
I20260807 18:12:23.676285 21453 tablet_replica.cc:330] T 63f39289a0524e3c814ff266684ffd82 P 36c9b6beec1645a1b0a94a0850f35ad0: stopping tablet replica
I20260807 18:12:23.676800 21453 raft_consensus.cc:2229] T 63f39289a0524e3c814ff266684ffd82 P 36c9b6beec1645a1b0a94a0850f35ad0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:12:23.677357 21453 raft_consensus.cc:2258] T 63f39289a0524e3c814ff266684ffd82 P 36c9b6beec1645a1b0a94a0850f35ad0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:12:23.677551 21424 tablet_bootstrap.cc:492] T 33b7d5507e7a40b88cef2e6e33bdad1b P 4be4e3b09afc4a4d821fad7dc6b3bd87: No bootstrap required, opened a new log
I20260807 18:12:23.677758 21424 ts_tablet_manager.cc:1397] T 33b7d5507e7a40b88cef2e6e33bdad1b P 4be4e3b09afc4a4d821fad7dc6b3bd87: Time spent bootstrapping tablet: real 0.012s user 0.003s sys 0.006s
I20260807 18:12:23.678690 21424 raft_consensus.cc:348] T 33b7d5507e7a40b88cef2e6e33bdad1b P 4be4e3b09afc4a4d821fad7dc6b3bd87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4be4e3b09afc4a4d821fad7dc6b3bd87" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 34831 } } peers { permanent_uuid: "36c9b6beec1645a1b0a94a0850f35ad0" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 40867 } } peers { permanent_uuid: "99aa7ef775274a3db8e5f5704b9105a0" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 39957 } }
I20260807 18:12:23.678987 21424 raft_consensus.cc:374] T 33b7d5507e7a40b88cef2e6e33bdad1b P 4be4e3b09afc4a4d821fad7dc6b3bd87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:12:23.679271 21424 raft_consensus.cc:729] T 33b7d5507e7a40b88cef2e6e33bdad1b P 4be4e3b09afc4a4d821fad7dc6b3bd87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4be4e3b09afc4a4d821fad7dc6b3bd87, State: Initialized, Role: FOLLOWER
I20260807 18:12:23.679661 21453 ts_tablet_manager.cc:1905] T 63f39289a0524e3c814ff266684ffd82 P 36c9b6beec1645a1b0a94a0850f35ad0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:12:23.679672 21424 consensus_queue.cc:260] T 33b7d5507e7a40b88cef2e6e33bdad1b P 4be4e3b09afc4a4d821fad7dc6b3bd87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4be4e3b09afc4a4d821fad7dc6b3bd87" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 34831 } } peers { permanent_uuid: "36c9b6beec1645a1b0a94a0850f35ad0" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 40867 } } peers { permanent_uuid: "99aa7ef775274a3db8e5f5704b9105a0" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 39957 } }
I20260807 18:12:23.680406 21424 ts_tablet_manager.cc:1428] T 33b7d5507e7a40b88cef2e6e33bdad1b P 4be4e3b09afc4a4d821fad7dc6b3bd87: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260807 18:12:23.687654 21453 ts_tablet_manager.cc:1918] T 63f39289a0524e3c814ff266684ffd82 P 36c9b6beec1645a1b0a94a0850f35ad0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:12:23.687865 21453 log.cc:1196] T 63f39289a0524e3c814ff266684ffd82 P 36c9b6beec1645a1b0a94a0850f35ad0: Deleting WAL directory at /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126328808105-19048-0/minicluster-data/ts-1/wal/wals/63f39289a0524e3c814ff266684ffd82
I20260807 18:12:23.688473 21453 ts_tablet_manager.cc:1939] T 63f39289a0524e3c814ff266684ffd82 P 36c9b6beec1645a1b0a94a0850f35ad0: Deleting consensus metadata
I20260807 18:12:23.689239 21429 raft_consensus.cc:482] T 33b7d5507e7a40b88cef2e6e33bdad1b P 99aa7ef775274a3db8e5f5704b9105a0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:12:23.689476 21429 raft_consensus.cc:504] T 33b7d5507e7a40b88cef2e6e33bdad1b P 99aa7ef775274a3db8e5f5704b9105a0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4be4e3b09afc4a4d821fad7dc6b3bd87" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 34831 } } peers { permanent_uuid: "36c9b6beec1645a1b0a94a0850f35ad0" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 40867 } } peers { permanent_uuid: "99aa7ef775274a3db8e5f5704b9105a0" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 39957 } }
I20260807 18:12:23.690574 20658 catalog_manager.cc:4841] TS 36c9b6beec1645a1b0a94a0850f35ad0 (127.18.154.2:40867): tablet 63f39289a0524e3c814ff266684ffd82 (table test-table [id=145678dd8f904991953c7967fc49e8b7]) successfully deleted
I20260807 18:12:23.691272 21429 leader_election.cc:288] T 33b7d5507e7a40b88cef2e6e33bdad1b P 99aa7ef775274a3db8e5f5704b9105a0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4be4e3b09afc4a4d821fad7dc6b3bd87 (127.18.154.1:34831), 36c9b6beec1645a1b0a94a0850f35ad0 (127.18.154.2:40867)
I20260807 18:12:23.699168 21410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:23.691961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.3:58363 (local address 127.18.154.2:40867)
0807 18:12:23.692121 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0807 18:12:23.692127 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:12:23.692198 (+ 71us) server_negotiation.cc:407] Connection header received
0807 18:12:23.692521 (+ 323us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:12:23.692527 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:12:23.692605 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:12:23.692672 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:12:23.693391 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:23.694485 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:23.696719 (+ 2234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:23.698111 (+ 1392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:23.698291 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:23.698373 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0807 18:12:23.698718 (+ 345us) server_negotiation.cc:299] Negotiation successful
0807 18:12:23.698856 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 18:12:23.699790 21459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:23.691513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.2:40867 (local address 127.18.154.3:58363)
0807 18:12:23.692101 (+ 588us) negotiation.cc:107] Waiting for socket to connect
0807 18:12:23.692121 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 18:12:23.692322 (+ 201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:12:23.692881 (+ 559us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:12:23.692888 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:12:23.692915 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:12:23.693156 (+ 241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:12:23.693164 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:12:23.694647 (+ 1483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:12:23.694651 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:12:23.696541 (+ 1890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:12:23.696556 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:12:23.698305 (+ 1749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:12:23.698312 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:12:23.698506 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:23.698544 (+ 38us) client_negotiation.cc:769] Sending connection context
0807 18:12:23.699369 (+ 825us) client_negotiation.cc:240] Negotiation successful
0807 18:12:23.699510 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":454,"mutex_wait_us":25,"thread_start_us":633,"threads_started":1}
I20260807 18:12:23.700053 21418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:23.692163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.3:38801 (local address 127.18.154.1:34831)
0807 18:12:23.692391 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0807 18:12:23.692397 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:12:23.692819 (+ 422us) server_negotiation.cc:407] Connection header received
0807 18:12:23.692969 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:12:23.692976 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:12:23.693105 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:12:23.693238 (+ 133us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:12:23.693923 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:23.695201 (+ 1278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:23.697625 (+ 2424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:23.698683 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:23.698799 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:23.699086 (+ 287us) server_negotiation.cc:1035] Waiting for connection context
0807 18:12:23.699542 (+ 456us) server_negotiation.cc:299] Negotiation successful
0807 18:12:23.699713 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260807 18:12:23.700525 21126 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "33b7d5507e7a40b88cef2e6e33bdad1b" candidate_uuid: "99aa7ef775274a3db8e5f5704b9105a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36c9b6beec1645a1b0a94a0850f35ad0" is_pre_election: true
I20260807 18:12:23.700773 21126 raft_consensus.cc:2454] T 33b7d5507e7a40b88cef2e6e33bdad1b P 36c9b6beec1645a1b0a94a0850f35ad0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99aa7ef775274a3db8e5f5704b9105a0 in term 0.
I20260807 18:12:23.700855 21460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:23.691572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.1:34831 (local address 127.18.154.3:38801)
0807 18:12:23.692707 (+ 1135us) negotiation.cc:107] Waiting for socket to connect
0807 18:12:23.692725 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 18:12:23.692840 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:12:23.693255 (+ 415us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:12:23.693262 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:12:23.693293 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:12:23.693738 (+ 445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:12:23.693752 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:12:23.695351 (+ 1599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:12:23.695356 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:12:23.697222 (+ 1866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:12:23.697236 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:12:23.699080 (+ 1844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:12:23.699087 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:12:23.699297 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:23.699366 (+ 69us) client_negotiation.cc:769] Sending connection context
0807 18:12:23.700299 (+ 933us) client_negotiation.cc:240] Negotiation successful
0807 18:12:23.700450 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1016,"mutex_wait_us":1,"thread_start_us":899,"threads_started":1}
I20260807 18:12:23.701260 21226 leader_election.cc:302] T 33b7d5507e7a40b88cef2e6e33bdad1b P 99aa7ef775274a3db8e5f5704b9105a0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 36c9b6beec1645a1b0a94a0850f35ad0, 99aa7ef775274a3db8e5f5704b9105a0; no voters:
I20260807 18:12:23.701711 21429 raft_consensus.cc:2781] T 33b7d5507e7a40b88cef2e6e33bdad1b P 99aa7ef775274a3db8e5f5704b9105a0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:12:23.701864 21429 raft_consensus.cc:482] T 33b7d5507e7a40b88cef2e6e33bdad1b P 99aa7ef775274a3db8e5f5704b9105a0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:12:23.701973 21429 raft_consensus.cc:3037] T 33b7d5507e7a40b88cef2e6e33bdad1b P 99aa7ef775274a3db8e5f5704b9105a0 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:12:23.705585 21429 raft_consensus.cc:504] T 33b7d5507e7a40b88cef2e6e33bdad1b P 99aa7ef775274a3db8e5f5704b9105a0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4be4e3b09afc4a4d821fad7dc6b3bd87" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 34831 } } peers { permanent_uuid: "36c9b6beec1645a1b0a94a0850f35ad0" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 40867 } } peers { permanent_uuid: "99aa7ef775274a3db8e5f5704b9105a0" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 39957 } }
I20260807 18:12:23.705513 20916 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "33b7d5507e7a40b88cef2e6e33bdad1b" candidate_uuid: "99aa7ef775274a3db8e5f5704b9105a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4be4e3b09afc4a4d821fad7dc6b3bd87" is_pre_election: true
I20260807 18:12:23.706079 20916 raft_consensus.cc:2454] T 33b7d5507e7a40b88cef2e6e33bdad1b P 4be4e3b09afc4a4d821fad7dc6b3bd87 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99aa7ef775274a3db8e5f5704b9105a0 in term 0.
I20260807 18:12:23.706727 21429 leader_election.cc:288] T 33b7d5507e7a40b88cef2e6e33bdad1b P 99aa7ef775274a3db8e5f5704b9105a0 [CANDIDATE]: Term 1 election: Requested vote from peers 4be4e3b09afc4a4d821fad7dc6b3bd87 (127.18.154.1:34831), 36c9b6beec1645a1b0a94a0850f35ad0 (127.18.154.2:40867)
I20260807 18:12:23.706750 20916 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "33b7d5507e7a40b88cef2e6e33bdad1b" candidate_uuid: "99aa7ef775274a3db8e5f5704b9105a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4be4e3b09afc4a4d821fad7dc6b3bd87"
I20260807 18:12:23.706943 20916 raft_consensus.cc:3037] T 33b7d5507e7a40b88cef2e6e33bdad1b P 4be4e3b09afc4a4d821fad7dc6b3bd87 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:12:23.707594 21126 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "33b7d5507e7a40b88cef2e6e33bdad1b" candidate_uuid: "99aa7ef775274a3db8e5f5704b9105a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36c9b6beec1645a1b0a94a0850f35ad0"
I20260807 18:12:23.707805 21126 raft_consensus.cc:3037] T 33b7d5507e7a40b88cef2e6e33bdad1b P 36c9b6beec1645a1b0a94a0850f35ad0 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:12:23.709844 20916 raft_consensus.cc:2454] T 33b7d5507e7a40b88cef2e6e33bdad1b P 4be4e3b09afc4a4d821fad7dc6b3bd87 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99aa7ef775274a3db8e5f5704b9105a0 in term 1.
I20260807 18:12:23.710299 21225 leader_election.cc:302] T 33b7d5507e7a40b88cef2e6e33bdad1b P 99aa7ef775274a3db8e5f5704b9105a0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4be4e3b09afc4a4d821fad7dc6b3bd87, 99aa7ef775274a3db8e5f5704b9105a0; no voters:
I20260807 18:12:23.710606 21429 raft_consensus.cc:2781] T 33b7d5507e7a40b88cef2e6e33bdad1b P 99aa7ef775274a3db8e5f5704b9105a0 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:12:23.711127 21126 raft_consensus.cc:2454] T 33b7d5507e7a40b88cef2e6e33bdad1b P 36c9b6beec1645a1b0a94a0850f35ad0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99aa7ef775274a3db8e5f5704b9105a0 in term 1.
I20260807 18:12:23.711740 21429 raft_consensus.cc:686] T 33b7d5507e7a40b88cef2e6e33bdad1b P 99aa7ef775274a3db8e5f5704b9105a0 [term 1 LEADER]: Becoming Leader. State: Replica: 99aa7ef775274a3db8e5f5704b9105a0, State: Running, Role: LEADER
I20260807 18:12:23.712173 21429 consensus_queue.cc:237] T 33b7d5507e7a40b88cef2e6e33bdad1b P 99aa7ef775274a3db8e5f5704b9105a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4be4e3b09afc4a4d821fad7dc6b3bd87" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 34831 } } peers { permanent_uuid: "36c9b6beec1645a1b0a94a0850f35ad0" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 40867 } } peers { permanent_uuid: "99aa7ef775274a3db8e5f5704b9105a0" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 39957 } }
I20260807 18:12:23.716080 20681 catalog_manager.cc:5495] T 33b7d5507e7a40b88cef2e6e33bdad1b P 99aa7ef775274a3db8e5f5704b9105a0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 99aa7ef775274a3db8e5f5704b9105a0 (127.18.154.3). New cstate: current_term: 1 leader_uuid: "99aa7ef775274a3db8e5f5704b9105a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4be4e3b09afc4a4d821fad7dc6b3bd87" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 34831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "36c9b6beec1645a1b0a94a0850f35ad0" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 40867 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "99aa7ef775274a3db8e5f5704b9105a0" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 39957 } health_report { overall_health: HEALTHY } } }
Aug 07 18:12:23 dist-test-slave-gkw7 krb5kdc[20611](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126341, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.62@KRBTEST.COM for kudu/127.18.154.62@KRBTEST.COM
I20260807 18:12:23.760908 21478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:23.744279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43098 (local address 127.18.154.62:40323)
0807 18:12:23.745178 (+ 899us) server_negotiation.cc:206] Beginning negotiation
0807 18:12:23.745183 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:12:23.745202 (+ 19us) server_negotiation.cc:407] Connection header received
0807 18:12:23.745252 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:12:23.745256 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:12:23.745330 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:12:23.745621 (+ 291us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:12:23.747208 (+ 1587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:23.748016 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:23.750153 (+ 2137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:23.750646 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:23.750777 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:23.755209 (+ 4432us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:12:23.755213 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:12:23.755217 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:12:23.755245 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:12:23.758067 (+ 2822us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:12:23.758788 (+ 721us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:12:23.758796 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:12:23.758804 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:12:23.758926 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:12:23.759333 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:12:23.759340 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:12:23.759346 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:12:23.759616 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:12:23.759782 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0807 18:12:23.760185 (+ 403us) server_negotiation.cc:299] Negotiation successful
0807 18:12:23.760453 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":795,"thread_start_us":643,"threads_started":1}
I20260807 18:12:23.760969 21476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:23.744095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:40323 (local address 127.0.0.1:43098)
0807 18:12:23.744786 (+ 691us) negotiation.cc:107] Waiting for socket to connect
0807 18:12:23.744814 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0807 18:12:23.744973 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:12:23.745644 (+ 671us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:12:23.745649 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:12:23.746692 (+ 1043us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:12:23.746990 (+ 298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:12:23.747005 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:12:23.748206 (+ 1201us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:12:23.748213 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:12:23.749977 (+ 1764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:12:23.749994 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:12:23.750867 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:12:23.750875 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:12:23.751156 (+ 281us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:23.751938 (+ 782us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:12:23.751966 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:12:23.755000 (+ 3034us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:12:23.758203 (+ 3203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:12:23.758219 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:12:23.758228 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:12:23.758654 (+ 426us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:12:23.759048 (+ 394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:12:23.759053 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:12:23.759056 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:12:23.759221 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:12:23.759751 (+ 530us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:12:23.759760 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:12:23.759935 (+ 175us) client_negotiation.cc:769] Sending connection context
0807 18:12:23.760109 (+ 174us) client_negotiation.cc:240] Negotiation successful
0807 18:12:23.760570 (+ 461us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":506,"mutex_wait_us":185,"thread_start_us":363,"threads_started":1}
I20260807 18:12:23.767305 20681 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.18.154.62@KRBTEST.COM'} at 127.0.0.1:43098:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260807 18:12:23.768515 20681 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:12:23.775733 20881 tablet_service.cc:1467] Processing CreateTablet for tablet 49deb38ba24d48baa56143969f8217f3 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=cb80553c59384d66ac2f985df51e0bb2]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 18:12:23.775924 21294 tablet_service.cc:1467] Processing CreateTablet for tablet 49deb38ba24d48baa56143969f8217f3 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=cb80553c59384d66ac2f985df51e0bb2]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 18:12:23.776103 21098 tablet_service.cc:1467] Processing CreateTablet for tablet 49deb38ba24d48baa56143969f8217f3 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=cb80553c59384d66ac2f985df51e0bb2]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 18:12:23.776441 20881 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 49deb38ba24d48baa56143969f8217f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:12:23.776536 21294 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 49deb38ba24d48baa56143969f8217f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:12:23.776738 21098 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 49deb38ba24d48baa56143969f8217f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:12:23.784598 21419 tablet_bootstrap.cc:492] T 49deb38ba24d48baa56143969f8217f3 P 99aa7ef775274a3db8e5f5704b9105a0: Bootstrap starting.
I20260807 18:12:23.785091 21424 tablet_bootstrap.cc:492] T 49deb38ba24d48baa56143969f8217f3 P 4be4e3b09afc4a4d821fad7dc6b3bd87: Bootstrap starting.
I20260807 18:12:23.786258 21419 tablet_bootstrap.cc:654] T 49deb38ba24d48baa56143969f8217f3 P 99aa7ef775274a3db8e5f5704b9105a0: Neither blocks nor log segments found. Creating new log.
I20260807 18:12:23.787014 21424 tablet_bootstrap.cc:654] T 49deb38ba24d48baa56143969f8217f3 P 4be4e3b09afc4a4d821fad7dc6b3bd87: Neither blocks nor log segments found. Creating new log.
I20260807 18:12:23.788164 21423 tablet_bootstrap.cc:492] T 49deb38ba24d48baa56143969f8217f3 P 36c9b6beec1645a1b0a94a0850f35ad0: Bootstrap starting.
I20260807 18:12:23.788931 21419 tablet_bootstrap.cc:492] T 49deb38ba24d48baa56143969f8217f3 P 99aa7ef775274a3db8e5f5704b9105a0: No bootstrap required, opened a new log
I20260807 18:12:23.789060 21419 ts_tablet_manager.cc:1397] T 49deb38ba24d48baa56143969f8217f3 P 99aa7ef775274a3db8e5f5704b9105a0: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260807 18:12:23.789739 21419 raft_consensus.cc:348] T 49deb38ba24d48baa56143969f8217f3 P 99aa7ef775274a3db8e5f5704b9105a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4be4e3b09afc4a4d821fad7dc6b3bd87" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 34831 } } peers { permanent_uuid: "99aa7ef775274a3db8e5f5704b9105a0" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 39957 } } peers { permanent_uuid: "36c9b6beec1645a1b0a94a0850f35ad0" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 40867 } }
I20260807 18:12:23.790012 21419 raft_consensus.cc:374] T 49deb38ba24d48baa56143969f8217f3 P 99aa7ef775274a3db8e5f5704b9105a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:12:23.790094 21419 raft_consensus.cc:729] T 49deb38ba24d48baa56143969f8217f3 P 99aa7ef775274a3db8e5f5704b9105a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99aa7ef775274a3db8e5f5704b9105a0, State: Initialized, Role: FOLLOWER
I20260807 18:12:23.790091 21423 tablet_bootstrap.cc:654] T 49deb38ba24d48baa56143969f8217f3 P 36c9b6beec1645a1b0a94a0850f35ad0: Neither blocks nor log segments found. Creating new log.
I20260807 18:12:23.790329 21419 consensus_queue.cc:260] T 49deb38ba24d48baa56143969f8217f3 P 99aa7ef775274a3db8e5f5704b9105a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4be4e3b09afc4a4d821fad7dc6b3bd87" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 34831 } } peers { permanent_uuid: "99aa7ef775274a3db8e5f5704b9105a0" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 39957 } } peers { permanent_uuid: "36c9b6beec1645a1b0a94a0850f35ad0" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 40867 } }
I20260807 18:12:23.790913 21419 ts_tablet_manager.cc:1428] T 49deb38ba24d48baa56143969f8217f3 P 99aa7ef775274a3db8e5f5704b9105a0: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260807 18:12:23.790784 21429 tablet_replica.cc:439] T 49deb38ba24d48baa56143969f8217f3 P 99aa7ef775274a3db8e5f5704b9105a0: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4be4e3b09afc4a4d821fad7dc6b3bd87" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 34831 } } peers { permanent_uuid: "99aa7ef775274a3db8e5f5704b9105a0" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 39957 } } peers { permanent_uuid: "36c9b6beec1645a1b0a94a0850f35ad0" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 40867 } } }
I20260807 18:12:23.791092 21429 tablet_replica.cc:442] T 49deb38ba24d48baa56143969f8217f3 P 99aa7ef775274a3db8e5f5704b9105a0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 18:12:23.795172 21423 tablet_bootstrap.cc:492] T 49deb38ba24d48baa56143969f8217f3 P 36c9b6beec1645a1b0a94a0850f35ad0: No bootstrap required, opened a new log
I20260807 18:12:23.795392 21423 ts_tablet_manager.cc:1397] T 49deb38ba24d48baa56143969f8217f3 P 36c9b6beec1645a1b0a94a0850f35ad0: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.002s
I20260807 18:12:23.796173 21423 raft_consensus.cc:348] T 49deb38ba24d48baa56143969f8217f3 P 36c9b6beec1645a1b0a94a0850f35ad0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4be4e3b09afc4a4d821fad7dc6b3bd87" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 34831 } } peers { permanent_uuid: "99aa7ef775274a3db8e5f5704b9105a0" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 39957 } } peers { permanent_uuid: "36c9b6beec1645a1b0a94a0850f35ad0" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 40867 } }
I20260807 18:12:23.796403 21423 raft_consensus.cc:374] T 49deb38ba24d48baa56143969f8217f3 P 36c9b6beec1645a1b0a94a0850f35ad0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:12:23.796514 21423 raft_consensus.cc:729] T 49deb38ba24d48baa56143969f8217f3 P 36c9b6beec1645a1b0a94a0850f35ad0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36c9b6beec1645a1b0a94a0850f35ad0, State: Initialized, Role: FOLLOWER
I20260807 18:12:23.796806 21423 consensus_queue.cc:260] T 49deb38ba24d48baa56143969f8217f3 P 36c9b6beec1645a1b0a94a0850f35ad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4be4e3b09afc4a4d821fad7dc6b3bd87" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 34831 } } peers { permanent_uuid: "99aa7ef775274a3db8e5f5704b9105a0" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 39957 } } peers { permanent_uuid: "36c9b6beec1645a1b0a94a0850f35ad0" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 40867 } }
I20260807 18:12:23.797437 21423 ts_tablet_manager.cc:1428] T 49deb38ba24d48baa56143969f8217f3 P 36c9b6beec1645a1b0a94a0850f35ad0: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260807 18:12:23.797298 21431 tablet_replica.cc:439] T 49deb38ba24d48baa56143969f8217f3 P 36c9b6beec1645a1b0a94a0850f35ad0: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4be4e3b09afc4a4d821fad7dc6b3bd87" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 34831 } } peers { permanent_uuid: "99aa7ef775274a3db8e5f5704b9105a0" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 39957 } } peers { permanent_uuid: "36c9b6beec1645a1b0a94a0850f35ad0" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 40867 } } }
I20260807 18:12:23.797780 21431 tablet_replica.cc:442] T 49deb38ba24d48baa56143969f8217f3 P 36c9b6beec1645a1b0a94a0850f35ad0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 18:12:23.798429 21424 tablet_bootstrap.cc:492] T 49deb38ba24d48baa56143969f8217f3 P 4be4e3b09afc4a4d821fad7dc6b3bd87: No bootstrap required, opened a new log
I20260807 18:12:23.798636 21424 ts_tablet_manager.cc:1397] T 49deb38ba24d48baa56143969f8217f3 P 4be4e3b09afc4a4d821fad7dc6b3bd87: Time spent bootstrapping tablet: real 0.014s user 0.006s sys 0.000s
I20260807 18:12:23.799500 21424 raft_consensus.cc:348] T 49deb38ba24d48baa56143969f8217f3 P 4be4e3b09afc4a4d821fad7dc6b3bd87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4be4e3b09afc4a4d821fad7dc6b3bd87" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 34831 } } peers { permanent_uuid: "99aa7ef775274a3db8e5f5704b9105a0" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 39957 } } peers { permanent_uuid: "36c9b6beec1645a1b0a94a0850f35ad0" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 40867 } }
I20260807 18:12:23.799784 21424 raft_consensus.cc:374] T 49deb38ba24d48baa56143969f8217f3 P 4be4e3b09afc4a4d821fad7dc6b3bd87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:12:23.799876 21424 raft_consensus.cc:729] T 49deb38ba24d48baa56143969f8217f3 P 4be4e3b09afc4a4d821fad7dc6b3bd87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4be4e3b09afc4a4d821fad7dc6b3bd87, State: Initialized, Role: FOLLOWER
I20260807 18:12:23.800151 21424 consensus_queue.cc:260] T 49deb38ba24d48baa56143969f8217f3 P 4be4e3b09afc4a4d821fad7dc6b3bd87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4be4e3b09afc4a4d821fad7dc6b3bd87" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 34831 } } peers { permanent_uuid: "99aa7ef775274a3db8e5f5704b9105a0" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 39957 } } peers { permanent_uuid: "36c9b6beec1645a1b0a94a0850f35ad0" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 40867 } }
I20260807 18:12:23.800657 21430 tablet_replica.cc:439] T 49deb38ba24d48baa56143969f8217f3 P 4be4e3b09afc4a4d821fad7dc6b3bd87: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4be4e3b09afc4a4d821fad7dc6b3bd87" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 34831 } } peers { permanent_uuid: "99aa7ef775274a3db8e5f5704b9105a0" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 39957 } } peers { permanent_uuid: "36c9b6beec1645a1b0a94a0850f35ad0" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 40867 } } }
I20260807 18:12:23.800969 21430 tablet_replica.cc:442] T 49deb38ba24d48baa56143969f8217f3 P 4be4e3b09afc4a4d821fad7dc6b3bd87: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 18:12:23.802085 21424 ts_tablet_manager.cc:1428] T 49deb38ba24d48baa56143969f8217f3 P 4be4e3b09afc4a4d821fad7dc6b3bd87: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260807 18:12:23.957345 21429 raft_consensus.cc:482] T 49deb38ba24d48baa56143969f8217f3 P 99aa7ef775274a3db8e5f5704b9105a0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:12:23.957662 21429 raft_consensus.cc:504] T 49deb38ba24d48baa56143969f8217f3 P 99aa7ef775274a3db8e5f5704b9105a0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4be4e3b09afc4a4d821fad7dc6b3bd87" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 34831 } } peers { permanent_uuid: "99aa7ef775274a3db8e5f5704b9105a0" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 39957 } } peers { permanent_uuid: "36c9b6beec1645a1b0a94a0850f35ad0" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 40867 } }
I20260807 18:12:23.958379 21429 leader_election.cc:288] T 49deb38ba24d48baa56143969f8217f3 P 99aa7ef775274a3db8e5f5704b9105a0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4be4e3b09afc4a4d821fad7dc6b3bd87 (127.18.154.1:34831), 36c9b6beec1645a1b0a94a0850f35ad0 (127.18.154.2:40867)
I20260807 18:12:23.959030 21126 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "49deb38ba24d48baa56143969f8217f3" candidate_uuid: "99aa7ef775274a3db8e5f5704b9105a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36c9b6beec1645a1b0a94a0850f35ad0" is_pre_election: true
I20260807 18:12:23.959110 20916 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "49deb38ba24d48baa56143969f8217f3" candidate_uuid: "99aa7ef775274a3db8e5f5704b9105a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4be4e3b09afc4a4d821fad7dc6b3bd87" is_pre_election: true
I20260807 18:12:23.959345 21126 raft_consensus.cc:2454] T 49deb38ba24d48baa56143969f8217f3 P 36c9b6beec1645a1b0a94a0850f35ad0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99aa7ef775274a3db8e5f5704b9105a0 in term 0.
I20260807 18:12:23.959372 20916 raft_consensus.cc:2454] T 49deb38ba24d48baa56143969f8217f3 P 4be4e3b09afc4a4d821fad7dc6b3bd87 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99aa7ef775274a3db8e5f5704b9105a0 in term 0.
I20260807 18:12:23.959810 21226 leader_election.cc:302] T 49deb38ba24d48baa56143969f8217f3 P 99aa7ef775274a3db8e5f5704b9105a0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 36c9b6beec1645a1b0a94a0850f35ad0, 99aa7ef775274a3db8e5f5704b9105a0; no voters:
I20260807 18:12:23.960150 21429 raft_consensus.cc:2781] T 49deb38ba24d48baa56143969f8217f3 P 99aa7ef775274a3db8e5f5704b9105a0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:12:23.960263 21429 raft_consensus.cc:482] T 49deb38ba24d48baa56143969f8217f3 P 99aa7ef775274a3db8e5f5704b9105a0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:12:23.960439 21429 raft_consensus.cc:3037] T 49deb38ba24d48baa56143969f8217f3 P 99aa7ef775274a3db8e5f5704b9105a0 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:12:23.962733 21429 raft_consensus.cc:504] T 49deb38ba24d48baa56143969f8217f3 P 99aa7ef775274a3db8e5f5704b9105a0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4be4e3b09afc4a4d821fad7dc6b3bd87" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 34831 } } peers { permanent_uuid: "99aa7ef775274a3db8e5f5704b9105a0" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 39957 } } peers { permanent_uuid: "36c9b6beec1645a1b0a94a0850f35ad0" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 40867 } }
I20260807 18:12:23.963317 21429 leader_election.cc:288] T 49deb38ba24d48baa56143969f8217f3 P 99aa7ef775274a3db8e5f5704b9105a0 [CANDIDATE]: Term 1 election: Requested vote from peers 4be4e3b09afc4a4d821fad7dc6b3bd87 (127.18.154.1:34831), 36c9b6beec1645a1b0a94a0850f35ad0 (127.18.154.2:40867)
I20260807 18:12:23.963857 20916 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "49deb38ba24d48baa56143969f8217f3" candidate_uuid: "99aa7ef775274a3db8e5f5704b9105a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4be4e3b09afc4a4d821fad7dc6b3bd87"
I20260807 18:12:23.963871 21126 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "49deb38ba24d48baa56143969f8217f3" candidate_uuid: "99aa7ef775274a3db8e5f5704b9105a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36c9b6beec1645a1b0a94a0850f35ad0"
I20260807 18:12:23.964108 20916 raft_consensus.cc:3037] T 49deb38ba24d48baa56143969f8217f3 P 4be4e3b09afc4a4d821fad7dc6b3bd87 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:12:23.964138 21126 raft_consensus.cc:3037] T 49deb38ba24d48baa56143969f8217f3 P 36c9b6beec1645a1b0a94a0850f35ad0 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:12:23.966372 20916 raft_consensus.cc:2454] T 49deb38ba24d48baa56143969f8217f3 P 4be4e3b09afc4a4d821fad7dc6b3bd87 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99aa7ef775274a3db8e5f5704b9105a0 in term 1.
I20260807 18:12:23.966810 21225 leader_election.cc:302] T 49deb38ba24d48baa56143969f8217f3 P 99aa7ef775274a3db8e5f5704b9105a0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4be4e3b09afc4a4d821fad7dc6b3bd87, 99aa7ef775274a3db8e5f5704b9105a0; no voters:
I20260807 18:12:23.967078 21429 raft_consensus.cc:2781] T 49deb38ba24d48baa56143969f8217f3 P 99aa7ef775274a3db8e5f5704b9105a0 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:12:23.967267 21429 raft_consensus.cc:686] T 49deb38ba24d48baa56143969f8217f3 P 99aa7ef775274a3db8e5f5704b9105a0 [term 1 LEADER]: Becoming Leader. State: Replica: 99aa7ef775274a3db8e5f5704b9105a0, State: Running, Role: LEADER
I20260807 18:12:23.967566 21429 consensus_queue.cc:237] T 49deb38ba24d48baa56143969f8217f3 P 99aa7ef775274a3db8e5f5704b9105a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4be4e3b09afc4a4d821fad7dc6b3bd87" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 34831 } } peers { permanent_uuid: "99aa7ef775274a3db8e5f5704b9105a0" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 39957 } } peers { permanent_uuid: "36c9b6beec1645a1b0a94a0850f35ad0" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 40867 } }
I20260807 18:12:23.967624 21126 raft_consensus.cc:2454] T 49deb38ba24d48baa56143969f8217f3 P 36c9b6beec1645a1b0a94a0850f35ad0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99aa7ef775274a3db8e5f5704b9105a0 in term 1.
I20260807 18:12:23.969566 21462 tablet_replica.cc:439] T 49deb38ba24d48baa56143969f8217f3 P 99aa7ef775274a3db8e5f5704b9105a0: TxnStatusTablet state changed. Reason: New leader 99aa7ef775274a3db8e5f5704b9105a0. Latest consensus state: current_term: 1 leader_uuid: "99aa7ef775274a3db8e5f5704b9105a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4be4e3b09afc4a4d821fad7dc6b3bd87" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 34831 } } peers { permanent_uuid: "99aa7ef775274a3db8e5f5704b9105a0" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 39957 } } peers { permanent_uuid: "36c9b6beec1645a1b0a94a0850f35ad0" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 40867 } } }
I20260807 18:12:23.969985 21462 tablet_replica.cc:442] T 49deb38ba24d48baa56143969f8217f3 P 99aa7ef775274a3db8e5f5704b9105a0: This TxnStatusTablet replica's current role is: LEADER
I20260807 18:12:23.971728 21485 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260807 18:12:23.972115 21485 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260807 18:12:23.972146 20680 catalog_manager.cc:5495] T 49deb38ba24d48baa56143969f8217f3 P 99aa7ef775274a3db8e5f5704b9105a0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 99aa7ef775274a3db8e5f5704b9105a0 (127.18.154.3). New cstate: current_term: 1 leader_uuid: "99aa7ef775274a3db8e5f5704b9105a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4be4e3b09afc4a4d821fad7dc6b3bd87" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 34831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "99aa7ef775274a3db8e5f5704b9105a0" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 39957 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "36c9b6beec1645a1b0a94a0850f35ad0" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 40867 } health_report { overall_health: UNKNOWN } } }
I20260807 18:12:24.007602 21490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:23.998671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53344 (local address 127.18.154.3:39957)
0807 18:12:23.999930 (+ 1259us) server_negotiation.cc:206] Beginning negotiation
0807 18:12:23.999937 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:12:23.999959 (+ 22us) server_negotiation.cc:407] Connection header received
0807 18:12:24.000067 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:12:24.000073 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:12:24.000208 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:12:24.000388 (+ 180us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:12:24.000837 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:24.002168 (+ 1331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:24.004292 (+ 2124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:24.005007 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:24.005807 (+ 800us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:24.005953 (+ 146us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:12:24.006815 (+ 862us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:12:24.006974 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0807 18:12:24.007139 (+ 165us) server_negotiation.cc:299] Negotiation successful
0807 18:12:24.007232 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1132,"thread_start_us":975,"threads_started":1}
I20260807 18:12:24.007655 21476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:23.998711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.3:39957 (local address 127.0.0.1:53344)
0807 18:12:23.999133 (+ 422us) negotiation.cc:107] Waiting for socket to connect
0807 18:12:23.999157 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 18:12:23.999322 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:12:24.000440 (+ 1118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:12:24.000446 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:12:24.000461 (+ 15us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 18:12:24.000661 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:12:24.000673 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:12:24.002330 (+ 1657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:12:24.002336 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:12:24.004118 (+ 1782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:12:24.004136 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:12:24.005252 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:12:24.005259 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:12:24.005470 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:24.005596 (+ 126us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 18:12:24.006943 (+ 1347us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 18:12:24.006972 (+ 29us) client_negotiation.cc:769] Sending connection context
0807 18:12:24.007076 (+ 104us) client_negotiation.cc:240] Negotiation successful
0807 18:12:24.007250 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":247}
I20260807 18:12:24.013728 20916 raft_consensus.cc:1261] T 49deb38ba24d48baa56143969f8217f3 P 4be4e3b09afc4a4d821fad7dc6b3bd87 [term 1 FOLLOWER]: Refusing update from remote peer 99aa7ef775274a3db8e5f5704b9105a0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:12:24.013883 21126 raft_consensus.cc:1261] T 49deb38ba24d48baa56143969f8217f3 P 36c9b6beec1645a1b0a94a0850f35ad0 [term 1 FOLLOWER]: Refusing update from remote peer 99aa7ef775274a3db8e5f5704b9105a0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:12:24.014824 21462 consensus_queue.cc:1035] T 49deb38ba24d48baa56143969f8217f3 P 99aa7ef775274a3db8e5f5704b9105a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "36c9b6beec1645a1b0a94a0850f35ad0" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 40867 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:12:24.015280 21429 consensus_queue.cc:1035] T 49deb38ba24d48baa56143969f8217f3 P 99aa7ef775274a3db8e5f5704b9105a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4be4e3b09afc4a4d821fad7dc6b3bd87" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 34831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:12:24.029867 21431 tablet_replica.cc:439] T 49deb38ba24d48baa56143969f8217f3 P 36c9b6beec1645a1b0a94a0850f35ad0: TxnStatusTablet state changed. Reason: New leader 99aa7ef775274a3db8e5f5704b9105a0. Latest consensus state: current_term: 1 leader_uuid: "99aa7ef775274a3db8e5f5704b9105a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4be4e3b09afc4a4d821fad7dc6b3bd87" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 34831 } } peers { permanent_uuid: "99aa7ef775274a3db8e5f5704b9105a0" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 39957 } } peers { permanent_uuid: "36c9b6beec1645a1b0a94a0850f35ad0" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 40867 } } }
I20260807 18:12:24.030220 21431 tablet_replica.cc:442] T 49deb38ba24d48baa56143969f8217f3 P 36c9b6beec1645a1b0a94a0850f35ad0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 18:12:24.033115 21462 tablet_replica.cc:439] T 49deb38ba24d48baa56143969f8217f3 P 99aa7ef775274a3db8e5f5704b9105a0: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "99aa7ef775274a3db8e5f5704b9105a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4be4e3b09afc4a4d821fad7dc6b3bd87" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 34831 } } peers { permanent_uuid: "99aa7ef775274a3db8e5f5704b9105a0" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 39957 } } peers { permanent_uuid: "36c9b6beec1645a1b0a94a0850f35ad0" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 40867 } } }
I20260807 18:12:24.033409 21462 tablet_replica.cc:442] T 49deb38ba24d48baa56143969f8217f3 P 99aa7ef775274a3db8e5f5704b9105a0: This TxnStatusTablet replica's current role is: LEADER
I20260807 18:12:24.033645 21443 mvcc.cc:205] Tried to move back new op lower bound from 7315973505069780992 to 7315973504893571072. Current Snapshot: MvccSnapshot[applied={T|T < 7315973505069780992 or (T in {7315973505069780992})}]
I20260807 18:12:24.034108 21442 mvcc.cc:205] Tried to move back new op lower bound from 7315973505069780992 to 7315973504893571072. Current Snapshot: MvccSnapshot[applied={T|T < 7315973505069780992}]
I20260807 18:12:24.036574 21431 tablet_replica.cc:439] T 49deb38ba24d48baa56143969f8217f3 P 36c9b6beec1645a1b0a94a0850f35ad0: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "99aa7ef775274a3db8e5f5704b9105a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4be4e3b09afc4a4d821fad7dc6b3bd87" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 34831 } } peers { permanent_uuid: "99aa7ef775274a3db8e5f5704b9105a0" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 39957 } } peers { permanent_uuid: "36c9b6beec1645a1b0a94a0850f35ad0" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 40867 } } }
I20260807 18:12:24.036808 21431 tablet_replica.cc:442] T 49deb38ba24d48baa56143969f8217f3 P 36c9b6beec1645a1b0a94a0850f35ad0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 18:12:24.048394 21430 tablet_replica.cc:439] T 49deb38ba24d48baa56143969f8217f3 P 4be4e3b09afc4a4d821fad7dc6b3bd87: TxnStatusTablet state changed. Reason: New leader 99aa7ef775274a3db8e5f5704b9105a0. Latest consensus state: current_term: 1 leader_uuid: "99aa7ef775274a3db8e5f5704b9105a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4be4e3b09afc4a4d821fad7dc6b3bd87" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 34831 } } peers { permanent_uuid: "99aa7ef775274a3db8e5f5704b9105a0" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 39957 } } peers { permanent_uuid: "36c9b6beec1645a1b0a94a0850f35ad0" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 40867 } } }
I20260807 18:12:24.048815 21430 tablet_replica.cc:442] T 49deb38ba24d48baa56143969f8217f3 P 4be4e3b09afc4a4d821fad7dc6b3bd87: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 18:12:24.053202 21429 tablet_replica.cc:439] T 49deb38ba24d48baa56143969f8217f3 P 99aa7ef775274a3db8e5f5704b9105a0: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "99aa7ef775274a3db8e5f5704b9105a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4be4e3b09afc4a4d821fad7dc6b3bd87" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 34831 } } peers { permanent_uuid: "99aa7ef775274a3db8e5f5704b9105a0" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 39957 } } peers { permanent_uuid: "36c9b6beec1645a1b0a94a0850f35ad0" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 40867 } } }
I20260807 18:12:24.053623 21429 tablet_replica.cc:442] T 49deb38ba24d48baa56143969f8217f3 P 99aa7ef775274a3db8e5f5704b9105a0: This TxnStatusTablet replica's current role is: LEADER
I20260807 18:12:24.055580 21430 tablet_replica.cc:439] T 49deb38ba24d48baa56143969f8217f3 P 4be4e3b09afc4a4d821fad7dc6b3bd87: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "99aa7ef775274a3db8e5f5704b9105a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4be4e3b09afc4a4d821fad7dc6b3bd87" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 34831 } } peers { permanent_uuid: "99aa7ef775274a3db8e5f5704b9105a0" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 39957 } } peers { permanent_uuid: "36c9b6beec1645a1b0a94a0850f35ad0" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 40867 } } }
I20260807 18:12:24.055863 21430 tablet_replica.cc:442] T 49deb38ba24d48baa56143969f8217f3 P 4be4e3b09afc4a4d821fad7dc6b3bd87: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 18:12:24.060572 21490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:24.046448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53360 (local address 127.18.154.3:39957)
0807 18:12:24.046618 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0807 18:12:24.046624 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:12:24.046644 (+ 20us) server_negotiation.cc:407] Connection header received
0807 18:12:24.046725 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:12:24.046730 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:12:24.046822 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:12:24.047406 (+ 584us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:12:24.047553 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:24.048722 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:24.051195 (+ 2473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:24.051972 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:24.052788 (+ 816us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:24.058944 (+ 6156us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:12:24.059099 (+ 155us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:12:24.060117 (+ 1018us) server_negotiation.cc:1035] Waiting for connection context
0807 18:12:24.060168 (+ 51us) server_negotiation.cc:299] Negotiation successful
0807 18:12:24.060258 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260807 18:12:24.097870 21490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:24.091346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.3:46459 (local address 127.18.154.3:39957)
0807 18:12:24.092381 (+ 1035us) server_negotiation.cc:206] Beginning negotiation
0807 18:12:24.092388 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:12:24.092639 (+ 251us) server_negotiation.cc:407] Connection header received
0807 18:12:24.092862 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:12:24.092867 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:12:24.092960 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:12:24.093067 (+ 107us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:12:24.093646 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:24.094403 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:24.095947 (+ 1544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:24.096393 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:24.096531 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:24.096984 (+ 453us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:12:24.097090 (+ 106us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:12:24.097188 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0807 18:12:24.097315 (+ 127us) server_negotiation.cc:299] Negotiation successful
0807 18:12:24.097423 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":19,"server-negotiator.queue_time_us":869}
I20260807 18:12:24.097976 21497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:24.091411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.3:39957 (local address 127.18.154.3:46459)
0807 18:12:24.092467 (+ 1056us) negotiation.cc:107] Waiting for socket to connect
0807 18:12:24.092485 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 18:12:24.092734 (+ 249us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:12:24.093071 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:12:24.093075 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:12:24.093086 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 18:12:24.093470 (+ 384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:12:24.093479 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:12:24.094531 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:12:24.094539 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:12:24.095801 (+ 1262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:12:24.095813 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:12:24.096619 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:12:24.096625 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:12:24.096749 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:24.096802 (+ 53us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 18:12:24.097187 (+ 385us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 18:12:24.097199 (+ 12us) client_negotiation.cc:769] Sending connection context
0807 18:12:24.097298 (+ 99us) client_negotiation.cc:240] Negotiation successful
0807 18:12:24.097465 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":760,"mutex_wait_us":42,"thread_start_us":581,"threads_started":1}
I20260807 18:12:24.111245 20916 raft_consensus.cc:1261] T 33b7d5507e7a40b88cef2e6e33bdad1b P 4be4e3b09afc4a4d821fad7dc6b3bd87 [term 1 FOLLOWER]: Refusing update from remote peer 99aa7ef775274a3db8e5f5704b9105a0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:12:24.112442 21126 raft_consensus.cc:1261] T 33b7d5507e7a40b88cef2e6e33bdad1b P 36c9b6beec1645a1b0a94a0850f35ad0 [term 1 FOLLOWER]: Refusing update from remote peer 99aa7ef775274a3db8e5f5704b9105a0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:12:24.113111 21462 consensus_queue.cc:1035] T 33b7d5507e7a40b88cef2e6e33bdad1b P 99aa7ef775274a3db8e5f5704b9105a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4be4e3b09afc4a4d821fad7dc6b3bd87" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 34831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:12:24.113620 21462 consensus_queue.cc:1035] T 33b7d5507e7a40b88cef2e6e33bdad1b P 99aa7ef775274a3db8e5f5704b9105a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "36c9b6beec1645a1b0a94a0850f35ad0" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 40867 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:12:24.247871 20680 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43094:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:12:24.248323 20680 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43094:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:12:24.252645 20680 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 98661d04c87d4fd0834cfb4532531d75: Sending DeleteTablet for 3 replicas of tablet 33b7d5507e7a40b88cef2e6e33bdad1b
I20260807 18:12:24.253607 20881 tablet_service.cc:1514] Processing DeleteTablet for tablet 33b7d5507e7a40b88cef2e6e33bdad1b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:12:24 UTC) from {username='kudu', principal='kudu/127.18.154.62@KRBTEST.COM'} at 127.0.0.1:39402
I20260807 18:12:24.253641 21098 tablet_service.cc:1514] Processing DeleteTablet for tablet 33b7d5507e7a40b88cef2e6e33bdad1b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:12:24 UTC) from {username='kudu', principal='kudu/127.18.154.62@KRBTEST.COM'} at 127.0.0.1:50516
I20260807 18:12:24.254240 21512 tablet_replica.cc:330] T 33b7d5507e7a40b88cef2e6e33bdad1b P 36c9b6beec1645a1b0a94a0850f35ad0: stopping tablet replica
I20260807 18:12:24.254467 21512 raft_consensus.cc:2229] T 33b7d5507e7a40b88cef2e6e33bdad1b P 36c9b6beec1645a1b0a94a0850f35ad0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:12:24.254673 21510 tablet_replica.cc:330] T 33b7d5507e7a40b88cef2e6e33bdad1b P 4be4e3b09afc4a4d821fad7dc6b3bd87: stopping tablet replica
I20260807 18:12:24.254657 21293 tablet_service.cc:1514] Processing DeleteTablet for tablet 33b7d5507e7a40b88cef2e6e33bdad1b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:12:24 UTC) from {username='kudu', principal='kudu/127.18.154.62@KRBTEST.COM'} at 127.0.0.1:53334
I20260807 18:12:24.254746 21512 raft_consensus.cc:2258] T 33b7d5507e7a40b88cef2e6e33bdad1b P 36c9b6beec1645a1b0a94a0850f35ad0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:12:24.254866 21510 raft_consensus.cc:2229] T 33b7d5507e7a40b88cef2e6e33bdad1b P 4be4e3b09afc4a4d821fad7dc6b3bd87 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:12:24.255112 21510 raft_consensus.cc:2258] T 33b7d5507e7a40b88cef2e6e33bdad1b P 4be4e3b09afc4a4d821fad7dc6b3bd87 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:12:24.255959 21512 ts_tablet_manager.cc:1905] T 33b7d5507e7a40b88cef2e6e33bdad1b P 36c9b6beec1645a1b0a94a0850f35ad0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:12:24.263135 21510 ts_tablet_manager.cc:1905] T 33b7d5507e7a40b88cef2e6e33bdad1b P 4be4e3b09afc4a4d821fad7dc6b3bd87: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:12:24.265166 21513 tablet_replica.cc:330] T 33b7d5507e7a40b88cef2e6e33bdad1b P 99aa7ef775274a3db8e5f5704b9105a0: stopping tablet replica
I20260807 18:12:24.265563 21513 raft_consensus.cc:2229] T 33b7d5507e7a40b88cef2e6e33bdad1b P 99aa7ef775274a3db8e5f5704b9105a0 [term 1 LEADER]: Raft consensus shutting down.
I20260807 18:12:24.265822 21512 ts_tablet_manager.cc:1918] T 33b7d5507e7a40b88cef2e6e33bdad1b P 36c9b6beec1645a1b0a94a0850f35ad0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 18:12:24.266007 21512 log.cc:1196] T 33b7d5507e7a40b88cef2e6e33bdad1b P 36c9b6beec1645a1b0a94a0850f35ad0: Deleting WAL directory at /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126328808105-19048-0/minicluster-data/ts-1/wal/wals/33b7d5507e7a40b88cef2e6e33bdad1b
I20260807 18:12:24.266458 21513 raft_consensus.cc:2258] T 33b7d5507e7a40b88cef2e6e33bdad1b P 99aa7ef775274a3db8e5f5704b9105a0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:12:24.266564 21512 ts_tablet_manager.cc:1939] T 33b7d5507e7a40b88cef2e6e33bdad1b P 36c9b6beec1645a1b0a94a0850f35ad0: Deleting consensus metadata
I20260807 18:12:24.267738 21513 ts_tablet_manager.cc:1905] T 33b7d5507e7a40b88cef2e6e33bdad1b P 99aa7ef775274a3db8e5f5704b9105a0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:12:24.267974 20658 catalog_manager.cc:4841] TS 36c9b6beec1645a1b0a94a0850f35ad0 (127.18.154.2:40867): tablet 33b7d5507e7a40b88cef2e6e33bdad1b (table test-table [id=aff22c9b6da44025a4364dcc8a5ab141]) successfully deleted
I20260807 18:12:24.276446 21510 ts_tablet_manager.cc:1918] T 33b7d5507e7a40b88cef2e6e33bdad1b P 4be4e3b09afc4a4d821fad7dc6b3bd87: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 18:12:24.276793 21510 log.cc:1196] T 33b7d5507e7a40b88cef2e6e33bdad1b P 4be4e3b09afc4a4d821fad7dc6b3bd87: Deleting WAL directory at /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126328808105-19048-0/minicluster-data/ts-0/wal/wals/33b7d5507e7a40b88cef2e6e33bdad1b
I20260807 18:12:24.277877 21510 ts_tablet_manager.cc:1939] T 33b7d5507e7a40b88cef2e6e33bdad1b P 4be4e3b09afc4a4d821fad7dc6b3bd87: Deleting consensus metadata
I20260807 18:12:24.280184 20657 catalog_manager.cc:4841] TS 4be4e3b09afc4a4d821fad7dc6b3bd87 (127.18.154.1:34831): tablet 33b7d5507e7a40b88cef2e6e33bdad1b (table test-table [id=aff22c9b6da44025a4364dcc8a5ab141]) successfully deleted
I20260807 18:12:24.281175 21513 ts_tablet_manager.cc:1918] T 33b7d5507e7a40b88cef2e6e33bdad1b P 99aa7ef775274a3db8e5f5704b9105a0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 18:12:24.281894 21513 log.cc:1196] T 33b7d5507e7a40b88cef2e6e33bdad1b P 99aa7ef775274a3db8e5f5704b9105a0: Deleting WAL directory at /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786126328808105-19048-0/minicluster-data/ts-2/wal/wals/33b7d5507e7a40b88cef2e6e33bdad1b
I20260807 18:12:24.282548 21513 ts_tablet_manager.cc:1939] T 33b7d5507e7a40b88cef2e6e33bdad1b P 99aa7ef775274a3db8e5f5704b9105a0: Deleting consensus metadata
I20260807 18:12:24.283880 20656 catalog_manager.cc:4841] TS 99aa7ef775274a3db8e5f5704b9105a0 (127.18.154.3:39957): tablet 33b7d5507e7a40b88cef2e6e33bdad1b (table test-table [id=aff22c9b6da44025a4364dcc8a5ab141]) successfully deleted
Aug 07 18:12:24 dist-test-slave-gkw7 krb5kdc[20611](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126343, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.18.154.1@KRBTEST.COM
I20260807 18:12:24.307806 21517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:24.273824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39418 (local address 127.18.154.1:34831)
0807 18:12:24.286368 (+ 12544us) server_negotiation.cc:206] Beginning negotiation
0807 18:12:24.286377 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:12:24.286405 (+ 28us) server_negotiation.cc:407] Connection header received
0807 18:12:24.286498 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:12:24.286506 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:12:24.286668 (+ 162us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:12:24.286817 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:12:24.289468 (+ 2651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:24.291279 (+ 1811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:24.293864 (+ 2585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:24.294545 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:24.294689 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:24.300482 (+ 5793us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:12:24.300506 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:12:24.300528 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:12:24.300593 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:12:24.304474 (+ 3881us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:12:24.305288 (+ 814us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:12:24.305298 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:12:24.305302 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:12:24.305476 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:12:24.306023 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:12:24.306029 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:12:24.306032 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:12:24.306525 (+ 493us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:12:24.306857 (+ 332us) server_negotiation.cc:1035] Waiting for connection context
0807 18:12:24.307114 (+ 257us) server_negotiation.cc:299] Negotiation successful
0807 18:12:24.307355 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12335,"spinlock_wait_cycles":79232,"thread_start_us":12223,"threads_started":1}
W20260807 18:12:24.308688 20981 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39418
I20260807 18:12:24.326452 21523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:24.311770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43100 (local address 127.18.154.62:40323)
0807 18:12:24.312497 (+ 727us) server_negotiation.cc:206] Beginning negotiation
0807 18:12:24.312502 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:12:24.312517 (+ 15us) server_negotiation.cc:407] Connection header received
0807 18:12:24.312567 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:12:24.312571 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:12:24.312629 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:12:24.312753 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:12:24.314633 (+ 1880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:24.315483 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:24.316835 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:24.317344 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:24.317497 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:24.320229 (+ 2732us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:12:24.320244 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:12:24.320248 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:12:24.320286 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:12:24.323435 (+ 3149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:12:24.324188 (+ 753us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:12:24.324195 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:12:24.324201 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:12:24.324354 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:12:24.324776 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:12:24.324782 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:12:24.324786 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:12:24.325092 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:12:24.325377 (+ 285us) server_negotiation.cc:1035] Waiting for connection context
0807 18:12:24.325768 (+ 391us) server_negotiation.cc:299] Negotiation successful
0807 18:12:24.326026 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":573,"thread_start_us":369,"threads_started":1}
W20260807 18:12:24.327373 20680 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43100
I20260807 18:12:24.330240 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 20791
I20260807 18:12:24.346292 21517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:24.331785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39430 (local address 127.18.154.1:34831)
0807 18:12:24.332071 (+ 286us) server_negotiation.cc:206] Beginning negotiation
0807 18:12:24.332079 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:12:24.332194 (+ 115us) server_negotiation.cc:407] Connection header received
0807 18:12:24.332447 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:12:24.332452 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:12:24.332550 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:12:24.332664 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:12:24.334172 (+ 1508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:24.334924 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:24.336304 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:24.336769 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:24.336928 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:24.339423 (+ 2495us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:12:24.339430 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:12:24.339435 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:12:24.339485 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:12:24.343058 (+ 3573us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:12:24.343865 (+ 807us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:12:24.343873 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:12:24.343877 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:12:24.344121 (+ 244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:12:24.344602 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:12:24.344608 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:12:24.344612 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:12:24.344925 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:12:24.345237 (+ 312us) server_negotiation.cc:1035] Waiting for connection context
0807 18:12:24.345643 (+ 406us) server_negotiation.cc:299] Negotiation successful
0807 18:12:24.345901 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155}
W20260807 18:12:24.346966 20981 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39430
W20260807 18:12:24.347649 19048 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 20791: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 18:12:24.347930 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 20791
I20260807 18:12:24.392498 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 20993
Aug 07 18:12:24 dist-test-slave-gkw7 krb5kdc[20611](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126343, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.18.154.2@KRBTEST.COM
I20260807 18:12:24.410923 21525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:24.393587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50526 (local address 127.18.154.2:40867)
0807 18:12:24.394326 (+ 739us) server_negotiation.cc:206] Beginning negotiation
0807 18:12:24.394331 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:12:24.394351 (+ 20us) server_negotiation.cc:407] Connection header received
0807 18:12:24.394414 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:12:24.394418 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:12:24.394483 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:12:24.394592 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:12:24.396248 (+ 1656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:24.397203 (+ 955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:24.399038 (+ 1835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:24.399499 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:24.399610 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:24.404781 (+ 5171us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:12:24.404796 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:12:24.404810 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:12:24.404855 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:12:24.407940 (+ 3085us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:12:24.408666 (+ 726us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:12:24.408674 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:12:24.408676 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:12:24.408821 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:12:24.409235 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:12:24.409239 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:12:24.409241 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:12:24.409672 (+ 431us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:12:24.409985 (+ 313us) server_negotiation.cc:1035] Waiting for connection context
0807 18:12:24.410306 (+ 321us) server_negotiation.cc:299] Negotiation successful
0807 18:12:24.410548 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":606,"thread_start_us":462,"threads_started":1}
W20260807 18:12:24.411741 21188 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50526
W20260807 18:12:24.412513 19048 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 20993: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 18:12:24.412694 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 20993
I20260807 18:12:24.439652 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 21203
Aug 07 18:12:24 dist-test-slave-gkw7 krb5kdc[20611](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126343, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.18.154.3@KRBTEST.COM
I20260807 18:12:24.458514 21490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:24.440599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53372 (local address 127.18.154.3:39957)
0807 18:12:24.440808 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0807 18:12:24.440816 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:12:24.440847 (+ 31us) server_negotiation.cc:407] Connection header received
0807 18:12:24.440922 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:12:24.440927 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:12:24.441024 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:12:24.441161 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:12:24.442669 (+ 1508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:24.443864 (+ 1195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:24.445311 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:24.446020 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:24.446175 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:24.450757 (+ 4582us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:12:24.450773 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:12:24.450795 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:12:24.450849 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:12:24.455022 (+ 4173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:12:24.455988 (+ 966us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:12:24.455999 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:12:24.456002 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:12:24.456160 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:12:24.456603 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:12:24.456606 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:12:24.456608 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:12:24.457022 (+ 414us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:12:24.457333 (+ 311us) server_negotiation.cc:1035] Waiting for connection context
0807 18:12:24.457944 (+ 611us) server_negotiation.cc:299] Negotiation successful
0807 18:12:24.458149 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
W20260807 18:12:24.459239 21385 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53372
W20260807 18:12:24.459956 19048 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 21203: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 18:12:24.460136 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 21203
I20260807 18:12:24.489609 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 20635
I20260807 18:12:24.490826 20753 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:12:24.609316 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 20635
2026-08-07T18:12:24Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (5233 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126328808105-19048-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:12:24 dist-test-slave-gkw7 krb5kdc[21531](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:12:24 dist-test-slave-gkw7 krb5kdc[21531](info): set up 2 sockets
Aug 07 18:12:24 dist-test-slave-gkw7 krb5kdc[21531](info): commencing operation
krb5kdc: starting...
W20260807 18:12:25.982108 19048 mini_kdc.cc:121] Time spent starting KDC: real 1.321s user 0.001s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126328808105-19048-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126328808105-19048-0/krb5kdc/test-user.keytab.
Aug 07 18:12:26 dist-test-slave-gkw7 krb5kdc[21531](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126346, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:12:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:12:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.154.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.154.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.154.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.62.keytab.
Entry for principal kudu/127.18.154.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.62.keytab.
I20260807 18:12:26.205313 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126328808105-19048-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126328808105-19048-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126328808105-19048-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.154.62:43221
--webserver_interface=127.18.154.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.62.keytab
--principal=kudu/127.18.154.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.154.20:44629
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.154.62:43221
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126328808105-19048-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126328808105-19048-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126328808105-19048-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:12:26.544936 21549 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:12:26.545289 21549 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:12:26.545457 21549 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:12:26.552791 21549 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:12:26.552906 21549 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260807 18:12:26.552978 21549 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:12:26.553104 21549 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:12:26.553205 21549 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:12:26.553316 21549 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:12:26.563629 21549 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.154.20:44629
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126328808105-19048-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126328808105-19048-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.154.62:43221
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.62.keytab
--principal=kudu/127.18.154.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.154.62:43221
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.154.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126328808105-19048-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:12:26.564975 21549 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:12:26.566375 21549 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:12:26.578676 21555 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:12:26.578589 21556 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:12:26.578634 21559 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:12:26.579981 21549 server_base.cc:1034] running on GCE node
I20260807 18:12:26.580796 21549 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:12:26.582268 21549 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:12:26.583490 21549 hybrid_clock.cc:648] HybridClock initialized: now 1786126346583427 us; error 36 us; skew 500 ppm
I20260807 18:12:26.583846 21549 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:12:26 dist-test-slave-gkw7 krb5kdc[21531](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126346, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:12:26.587090 21549 init.cc:377] Logged in from keytab as kudu/127.18.154.62@KRBTEST.COM (short username kudu)
I20260807 18:12:26.588721 21549 webserver.cc:460] Webserver started at http://127.18.154.62:38855/ using document root <none> and password file <none>
I20260807 18:12:26.589475 21549 fs_manager.cc:359] Metadata directory not provided
I20260807 18:12:26.589632 21549 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:12:26.589977 21549 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:12:26.593003 21549 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126328808105-19048-0/minicluster-data/master-0/data/instance:
uuid: "6fff555ae1b94e1494e4fb470dad5674"
format_stamp: "Formatted at 2026-08-07 18:12:26 on dist-test-slave-gkw7"
server_key: "523268bb3a049f4622f7c4e91937df51"
server_key_iv: "57714c9b0b0b9e47ae29253f562cf1f3"
server_key_version: "encryptionkey@0"
I20260807 18:12:26.593873 21549 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126328808105-19048-0/minicluster-data/master-0/wal/instance:
uuid: "6fff555ae1b94e1494e4fb470dad5674"
format_stamp: "Formatted at 2026-08-07 18:12:26 on dist-test-slave-gkw7"
server_key: "523268bb3a049f4622f7c4e91937df51"
server_key_iv: "57714c9b0b0b9e47ae29253f562cf1f3"
server_key_version: "encryptionkey@0"
I20260807 18:12:26.599576 21549 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 18:12:26.603245 21570 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:12:26.604688 21549 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 18:12:26.604884 21549 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126328808105-19048-0/minicluster-data/master-0/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126328808105-19048-0/minicluster-data/master-0/wal
uuid: "6fff555ae1b94e1494e4fb470dad5674"
format_stamp: "Formatted at 2026-08-07 18:12:26 on dist-test-slave-gkw7"
server_key: "523268bb3a049f4622f7c4e91937df51"
server_key_iv: "57714c9b0b0b9e47ae29253f562cf1f3"
server_key_version: "encryptionkey@0"
I20260807 18:12:26.605113 21549 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126328808105-19048-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126328808105-19048-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126328808105-19048-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:12:26.623235 21549 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:12:26.627849 21549 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:12:26.628154 21549 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:12:26.651132 21549 rpc_server.cc:307] RPC server started. Bound to: 127.18.154.62:43221
I20260807 18:12:26.651160 21658 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.154.62:43221 every 8 connection(s)
I20260807 18:12:26.652590 21549 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
I20260807 18:12:26.656639 21659 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:12:26.657428 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 21549
I20260807 18:12:26.657712 19048 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126328808105-19048-0/minicluster-data/master-0/wal/instance
I20260807 18:12:26.658591 19048 external_mini_cluster.cc:1428] Setting key 78184291102eb56c08ddeec3331df57b
I20260807 18:12:26.672320 21659 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6fff555ae1b94e1494e4fb470dad5674: Bootstrap starting.
Aug 07 18:12:26 dist-test-slave-gkw7 krb5kdc[21531](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126346, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.154.62@KRBTEST.COM
I20260807 18:12:26.676433 21659 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6fff555ae1b94e1494e4fb470dad5674: Neither blocks nor log segments found. Creating new log.
I20260807 18:12:26.678025 21659 log.cc:824] T 00000000000000000000000000000000 P 6fff555ae1b94e1494e4fb470dad5674: Log is configured to *not* fsync() on all Append() calls
I20260807 18:12:26.681452 21659 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6fff555ae1b94e1494e4fb470dad5674: No bootstrap required, opened a new log
I20260807 18:12:26.684451 21662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:26.660862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58828 (local address 127.18.154.62:43221)
0807 18:12:26.661801 (+ 939us) server_negotiation.cc:206] Beginning negotiation
0807 18:12:26.661813 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 18:12:26.661862 (+ 49us) server_negotiation.cc:407] Connection header received
0807 18:12:26.663545 (+ 1683us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:12:26.663580 (+ 35us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:12:26.663983 (+ 403us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:12:26.664327 (+ 344us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:12:26.666451 (+ 2124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:26.667789 (+ 1338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:26.669795 (+ 2006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:26.670491 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:26.670624 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:26.675574 (+ 4950us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:12:26.675585 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:12:26.675607 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:12:26.675665 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:12:26.679700 (+ 4035us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:12:26.680480 (+ 780us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:12:26.680489 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:12:26.680497 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:12:26.680736 (+ 239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:12:26.681168 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:12:26.681173 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:12:26.681175 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:12:26.681951 (+ 776us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:12:26.682283 (+ 332us) server_negotiation.cc:1035] Waiting for connection context
0807 18:12:26.682618 (+ 335us) server_negotiation.cc:299] Negotiation successful
0807 18:12:26.683054 (+ 436us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":711,"thread_start_us":440,"threads_started":1}
I20260807 18:12:26.690140 21659 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6fff555ae1b94e1494e4fb470dad5674 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6fff555ae1b94e1494e4fb470dad5674" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 43221 } }
I20260807 18:12:26.690471 21659 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6fff555ae1b94e1494e4fb470dad5674 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:12:26.690544 21659 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6fff555ae1b94e1494e4fb470dad5674 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fff555ae1b94e1494e4fb470dad5674, State: Initialized, Role: FOLLOWER
I20260807 18:12:26.691056 21659 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6fff555ae1b94e1494e4fb470dad5674 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6fff555ae1b94e1494e4fb470dad5674" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 43221 } }
I20260807 18:12:26.691269 21659 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6fff555ae1b94e1494e4fb470dad5674 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:12:26.691367 21659 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6fff555ae1b94e1494e4fb470dad5674 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:12:26.691497 21659 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6fff555ae1b94e1494e4fb470dad5674 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:12:26.693684 21659 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6fff555ae1b94e1494e4fb470dad5674 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6fff555ae1b94e1494e4fb470dad5674" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 43221 } }
I20260807 18:12:26.694159 21659 leader_election.cc:302] T 00000000000000000000000000000000 P 6fff555ae1b94e1494e4fb470dad5674 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6fff555ae1b94e1494e4fb470dad5674; no voters:
I20260807 18:12:26.694682 21659 leader_election.cc:288] T 00000000000000000000000000000000 P 6fff555ae1b94e1494e4fb470dad5674 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:12:26.694861 21665 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6fff555ae1b94e1494e4fb470dad5674 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:12:26.695863 21665 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6fff555ae1b94e1494e4fb470dad5674 [term 1 LEADER]: Becoming Leader. State: Replica: 6fff555ae1b94e1494e4fb470dad5674, State: Running, Role: LEADER
I20260807 18:12:26.696548 21665 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6fff555ae1b94e1494e4fb470dad5674 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6fff555ae1b94e1494e4fb470dad5674" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 43221 } }
I20260807 18:12:26.697181 21659 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6fff555ae1b94e1494e4fb470dad5674 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:12:26.699780 21666 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6fff555ae1b94e1494e4fb470dad5674 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6fff555ae1b94e1494e4fb470dad5674" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6fff555ae1b94e1494e4fb470dad5674" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 43221 } } }
I20260807 18:12:26.699815 21667 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6fff555ae1b94e1494e4fb470dad5674 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6fff555ae1b94e1494e4fb470dad5674. Latest consensus state: current_term: 1 leader_uuid: "6fff555ae1b94e1494e4fb470dad5674" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6fff555ae1b94e1494e4fb470dad5674" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 43221 } } }
I20260807 18:12:26.700344 21666 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6fff555ae1b94e1494e4fb470dad5674 [sys.catalog]: This master's current role is: LEADER
I20260807 18:12:26.700352 21667 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6fff555ae1b94e1494e4fb470dad5674 [sys.catalog]: This master's current role is: LEADER
I20260807 18:12:26.701359 21676 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:12:26.708992 21676 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:12:26.717950 21676 catalog_manager.cc:1347] Generated new cluster ID: a055d5bfc5bc41fdbfc904568c2538da
I20260807 18:12:26.718078 21676 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:12:26.727830 21676 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:12:26.729148 21676 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:12:26.737136 21676 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6fff555ae1b94e1494e4fb470dad5674: Generated new TSK 0
I20260807 18:12:26.738109 21676 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.154.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.154.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.154.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.1.keytab.
Entry for principal kudu/127.18.154.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.1.keytab.
I20260807 18:12:26.791240 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126328808105-19048-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126328808105-19048-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126328808105-19048-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126328808105-19048-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.154.1:0
--local_ip_for_outbound_sockets=127.18.154.1
--webserver_interface=127.18.154.1
--webserver_port=0
--tserver_master_addrs=127.18.154.62:43221
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.1.keytab
--principal=kudu/127.18.154.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.154.20:44629
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126328808105-19048-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126328808105-19048-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126328808105-19048-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:12:27.132118 21695 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:12:27.132411 21695 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:12:27.132591 21695 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:12:27.139986 21695 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:12:27.140107 21695 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:12:27.140282 21695 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260807 18:12:27.140370 21695 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 18:12:27.140520 21695 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.154.1
I20260807 18:12:27.150682 21695 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.154.20:44629
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126328808105-19048-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126328808105-19048-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.1.keytab
--principal=kudu/127.18.154.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.154.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126328808105-19048-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.154.1
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.18.154.62:43221
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.154.1
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126328808105-19048-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:12:27.152155 21695 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:12:27.153661 21695 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:12:27.167578 21704 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:12:27.167502 21705 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:12:27.167554 21708 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:12:27.168237 21695 server_base.cc:1034] running on GCE node
I20260807 18:12:27.168963 21695 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:12:27.170083 21695 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:12:27.171296 21695 hybrid_clock.cc:648] HybridClock initialized: now 1786126347171248 us; error 35 us; skew 500 ppm
I20260807 18:12:27.171639 21695 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:12:27 dist-test-slave-gkw7 krb5kdc[21531](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126347, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:12:27.174943 21695 init.cc:377] Logged in from keytab as kudu/127.18.154.1@KRBTEST.COM (short username kudu)
I20260807 18:12:27.176581 21695 webserver.cc:460] Webserver started at http://127.18.154.1:34361/ using document root <none> and password file <none>
I20260807 18:12:27.177326 21695 fs_manager.cc:359] Metadata directory not provided
I20260807 18:12:27.177484 21695 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:12:27.177873 21695 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:12:27.181038 21695 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126328808105-19048-0/minicluster-data/ts-0/data/instance:
uuid: "f7ddbaeafa194da88d61994cf63503f6"
format_stamp: "Formatted at 2026-08-07 18:12:27 on dist-test-slave-gkw7"
server_key: "bfac08be191bbe5be859024218737429"
server_key_iv: "79b15380aa4b12a7a3e9e4b80ad79982"
server_key_version: "encryptionkey@0"
I20260807 18:12:27.181910 21695 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126328808105-19048-0/minicluster-data/ts-0/wal/instance:
uuid: "f7ddbaeafa194da88d61994cf63503f6"
format_stamp: "Formatted at 2026-08-07 18:12:27 on dist-test-slave-gkw7"
server_key: "bfac08be191bbe5be859024218737429"
server_key_iv: "79b15380aa4b12a7a3e9e4b80ad79982"
server_key_version: "encryptionkey@0"
I20260807 18:12:27.187619 21695 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 18:12:27.191313 21716 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:12:27.192756 21695 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 18:12:27.192950 21695 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126328808105-19048-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126328808105-19048-0/minicluster-data/ts-0/wal
uuid: "f7ddbaeafa194da88d61994cf63503f6"
format_stamp: "Formatted at 2026-08-07 18:12:27 on dist-test-slave-gkw7"
server_key: "bfac08be191bbe5be859024218737429"
server_key_iv: "79b15380aa4b12a7a3e9e4b80ad79982"
server_key_version: "encryptionkey@0"
I20260807 18:12:27.193164 21695 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126328808105-19048-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126328808105-19048-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126328808105-19048-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:12:27.210772 21695 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:12:27.214967 21695 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:12:27.215276 21695 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:12:27.218174 21695 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:12:27.218308 21695 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:12:27.218449 21695 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:12:27.218549 21695 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 18:12:27 dist-test-slave-gkw7 krb5kdc[21531](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126347, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.1@KRBTEST.COM for kudu/127.18.154.62@KRBTEST.COM
I20260807 18:12:27.242347 21728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:27.218686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.1:49099 (local address 127.18.154.62:43221)
0807 18:12:27.219511 (+ 825us) server_negotiation.cc:206] Beginning negotiation
0807 18:12:27.219517 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:12:27.220491 (+ 974us) server_negotiation.cc:407] Connection header received
0807 18:12:27.222778 (+ 2287us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:12:27.222784 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:12:27.222884 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:12:27.223009 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:12:27.225397 (+ 2388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:27.226780 (+ 1383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:27.229582 (+ 2802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:27.230025 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:27.230182 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:27.235523 (+ 5341us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:12:27.235532 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:12:27.235538 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:12:27.235585 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:12:27.238814 (+ 3229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:12:27.239848 (+ 1034us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:12:27.239857 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:12:27.239862 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:12:27.239999 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:12:27.240454 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:12:27.240460 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:12:27.240463 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:12:27.240760 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:12:27.240926 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0807 18:12:27.241711 (+ 785us) server_negotiation.cc:299] Negotiation successful
0807 18:12:27.241957 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":713,"thread_start_us":623,"threads_started":1}
I20260807 18:12:27.244042 21729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:27.219412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:43221 (local address 127.18.154.1:49099)
0807 18:12:27.220320 (+ 908us) negotiation.cc:107] Waiting for socket to connect
0807 18:12:27.220368 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 18:12:27.222408 (+ 2040us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:12:27.223315 (+ 907us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:12:27.223330 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:12:27.224255 (+ 925us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:12:27.225150 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:12:27.225171 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:12:27.226986 (+ 1815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:12:27.226993 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:12:27.229392 (+ 2399us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:12:27.229405 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:12:27.230304 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:12:27.230311 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:12:27.230653 (+ 342us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:27.231675 (+ 1022us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:12:27.231716 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:12:27.235187 (+ 3471us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:12:27.238982 (+ 3795us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:12:27.238991 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:12:27.239011 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:12:27.239709 (+ 698us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:12:27.240127 (+ 418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:12:27.240132 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:12:27.240136 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:12:27.240335 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:12:27.240907 (+ 572us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:12:27.240916 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:12:27.241298 (+ 382us) client_negotiation.cc:769] Sending connection context
0807 18:12:27.241639 (+ 341us) client_negotiation.cc:240] Negotiation successful
0807 18:12:27.242269 (+ 630us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":670,"thread_start_us":392,"threads_started":1}
I20260807 18:12:27.266547 21728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:27.251535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.1:36977 (local address 127.18.154.62:43221)
0807 18:12:27.251703 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0807 18:12:27.251708 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:12:27.252417 (+ 709us) server_negotiation.cc:407] Connection header received
0807 18:12:27.252623 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:12:27.252629 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:12:27.252706 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:12:27.252816 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:12:27.253996 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:27.254744 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:27.256104 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:27.256504 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:27.256643 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:27.259114 (+ 2471us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:12:27.259124 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:12:27.259132 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:12:27.259188 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:12:27.263181 (+ 3993us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:12:27.264114 (+ 933us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:12:27.264120 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:12:27.264124 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:12:27.264318 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:12:27.264883 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:12:27.264888 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:12:27.264891 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:12:27.265230 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:12:27.265430 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0807 18:12:27.266044 (+ 614us) server_negotiation.cc:299] Negotiation successful
0807 18:12:27.266302 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260807 18:12:27.267098 21801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:27.251626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:43221 (local address 127.18.154.1:36977)
0807 18:12:27.252301 (+ 675us) negotiation.cc:107] Waiting for socket to connect
0807 18:12:27.252321 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 18:12:27.252492 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:12:27.252851 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:12:27.252855 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:12:27.253407 (+ 552us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:12:27.253844 (+ 437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:12:27.253855 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:12:27.254885 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:12:27.254894 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:12:27.255941 (+ 1047us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:12:27.255952 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:12:27.256705 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:12:27.256713 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:12:27.256896 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:27.257707 (+ 811us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:12:27.257724 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:12:27.258879 (+ 1155us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:12:27.263327 (+ 4448us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:12:27.263333 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:12:27.263338 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:12:27.263979 (+ 641us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:12:27.264467 (+ 488us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:12:27.264475 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:12:27.264480 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:12:27.264709 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:12:27.265382 (+ 673us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:12:27.265390 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:12:27.265708 (+ 318us) client_negotiation.cc:769] Sending connection context
0807 18:12:27.265923 (+ 215us) client_negotiation.cc:240] Negotiation successful
0807 18:12:27.266422 (+ 499us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":550,"thread_start_us":401,"threads_started":1}
I20260807 18:12:27.284236 21695 rpc_server.cc:307] RPC server started. Bound to: 127.18.154.1:38089
I20260807 18:12:27.284276 21875 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.154.1:38089 every 8 connection(s)
I20260807 18:12:27.285650 21695 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126328808105-19048-0/minicluster-data/ts-0/data/info.pb
I20260807 18:12:27.288012 21877 heartbeater.cc:347] Connected to a master server at 127.18.154.62:43221
I20260807 18:12:27.288275 21877 heartbeater.cc:464] Registering TS with master...
I20260807 18:12:27.288967 21877 heartbeater.cc:511] Master 127.18.154.62:43221 requested a full tablet report, sending...
I20260807 18:12:27.291245 21594 ts_manager.cc:194] Registered new tserver with Master: f7ddbaeafa194da88d61994cf63503f6 (127.18.154.1:38089)
I20260807 18:12:27.292816 21594 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.154.1@KRBTEST.COM'} at 127.18.154.1:49099
I20260807 18:12:27.294051 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 21695
I20260807 18:12:27.294322 19048 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126328808105-19048-0/minicluster-data/ts-0/wal/instance
I20260807 18:12:27.295121 19048 external_mini_cluster.cc:1428] Setting key 9586229433319471c273286832595e03
WARNING: no policy specified for kudu/127.18.154.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.154.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.154.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.2.keytab.
Entry for principal kudu/127.18.154.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.2.keytab.
I20260807 18:12:27.344353 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126328808105-19048-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126328808105-19048-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126328808105-19048-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126328808105-19048-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.154.2:0
--local_ip_for_outbound_sockets=127.18.154.2
--webserver_interface=127.18.154.2
--webserver_port=0
--tserver_master_addrs=127.18.154.62:43221
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.2.keytab
--principal=kudu/127.18.154.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.154.20:44629
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126328808105-19048-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126328808105-19048-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126328808105-19048-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:12:27.682500 21881 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:12:27.682850 21881 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:12:27.683033 21881 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:12:27.690522 21881 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:12:27.690685 21881 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:12:27.690845 21881 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260807 18:12:27.690935 21881 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 18:12:27.691082 21881 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.154.2
I20260807 18:12:27.702054 21881 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.154.20:44629
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126328808105-19048-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126328808105-19048-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.2.keytab
--principal=kudu/127.18.154.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.154.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126328808105-19048-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.154.2
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.18.154.62:43221
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.154.2
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126328808105-19048-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:12:27.703492 21881 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:12:27.705001 21881 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:12:27.719544 21891 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:12:27.719761 21889 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:12:27.719589 21893 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:12:27.720481 21881 server_base.cc:1034] running on GCE node
I20260807 18:12:27.721110 21881 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:12:27.722193 21881 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:12:27.723445 21881 hybrid_clock.cc:648] HybridClock initialized: now 1786126347723389 us; error 45 us; skew 500 ppm
I20260807 18:12:27.723803 21881 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:12:27 dist-test-slave-gkw7 krb5kdc[21531](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126347, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:12:27.731900 21881 init.cc:377] Logged in from keytab as kudu/127.18.154.2@KRBTEST.COM (short username kudu)
I20260807 18:12:27.733616 21881 webserver.cc:460] Webserver started at http://127.18.154.2:46021/ using document root <none> and password file <none>
I20260807 18:12:27.734398 21881 fs_manager.cc:359] Metadata directory not provided
I20260807 18:12:27.734561 21881 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:12:27.734913 21881 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:12:27.738389 21881 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126328808105-19048-0/minicluster-data/ts-1/data/instance:
uuid: "bf17273192884a1e8a41b74403970a98"
format_stamp: "Formatted at 2026-08-07 18:12:27 on dist-test-slave-gkw7"
server_key: "7e6b4a6c5df7bf7c1699362224d5680d"
server_key_iv: "9bca9427fd5c43eb645e7b8fd1710d32"
server_key_version: "encryptionkey@0"
I20260807 18:12:27.739251 21881 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126328808105-19048-0/minicluster-data/ts-1/wal/instance:
uuid: "bf17273192884a1e8a41b74403970a98"
format_stamp: "Formatted at 2026-08-07 18:12:27 on dist-test-slave-gkw7"
server_key: "7e6b4a6c5df7bf7c1699362224d5680d"
server_key_iv: "9bca9427fd5c43eb645e7b8fd1710d32"
server_key_version: "encryptionkey@0"
I20260807 18:12:27.745214 21881 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.005s
I20260807 18:12:27.749212 21903 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:12:27.750635 21881 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:12:27.750823 21881 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126328808105-19048-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126328808105-19048-0/minicluster-data/ts-1/wal
uuid: "bf17273192884a1e8a41b74403970a98"
format_stamp: "Formatted at 2026-08-07 18:12:27 on dist-test-slave-gkw7"
server_key: "7e6b4a6c5df7bf7c1699362224d5680d"
server_key_iv: "9bca9427fd5c43eb645e7b8fd1710d32"
server_key_version: "encryptionkey@0"
I20260807 18:12:27.751034 21881 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126328808105-19048-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126328808105-19048-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126328808105-19048-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:12:27.787083 21881 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:12:27.791513 21881 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:12:27.791841 21881 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:12:27.794922 21881 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:12:27.795084 21881 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:12:27.795253 21881 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:12:27.795392 21881 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
Aug 07 18:12:27 dist-test-slave-gkw7 krb5kdc[21531](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126347, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.2@KRBTEST.COM for kudu/127.18.154.62@KRBTEST.COM
I20260807 18:12:27.824292 21913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:27.795526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.2:52599 (local address 127.18.154.62:43221)
0807 18:12:27.796428 (+ 902us) server_negotiation.cc:206] Beginning negotiation
0807 18:12:27.796435 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:12:27.797836 (+ 1401us) server_negotiation.cc:407] Connection header received
0807 18:12:27.801173 (+ 3337us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:12:27.801179 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:12:27.801283 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:12:27.801436 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:12:27.803952 (+ 2516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:27.804982 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:27.808557 (+ 3575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:27.809413 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:27.809563 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:27.816461 (+ 6898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:12:27.816471 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:12:27.816476 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:12:27.816530 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:12:27.820188 (+ 3658us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:12:27.821403 (+ 1215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:12:27.821409 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:12:27.821413 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:12:27.821638 (+ 225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:12:27.822330 (+ 692us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:12:27.822335 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:12:27.822339 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:12:27.822643 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:12:27.822831 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0807 18:12:27.823647 (+ 816us) server_negotiation.cc:299] Negotiation successful
0807 18:12:27.823892 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":769,"thread_start_us":673,"threads_started":1}
I20260807 18:12:27.825795 21914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:27.796395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:43221 (local address 127.18.154.2:52599)
0807 18:12:27.797474 (+ 1079us) negotiation.cc:107] Waiting for socket to connect
0807 18:12:27.797598 (+ 124us) client_negotiation.cc:174] Beginning negotiation
0807 18:12:27.800757 (+ 3159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:12:27.801765 (+ 1008us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:12:27.801789 (+ 24us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:12:27.802683 (+ 894us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:12:27.803734 (+ 1051us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:12:27.803774 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:12:27.805222 (+ 1448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:12:27.805230 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:12:27.808306 (+ 3076us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:12:27.808328 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:12:27.810101 (+ 1773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:12:27.810115 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:12:27.810535 (+ 420us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:27.811648 (+ 1113us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:12:27.811705 (+ 57us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:12:27.816025 (+ 4320us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:12:27.820366 (+ 4341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:12:27.820385 (+ 19us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:12:27.820412 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:12:27.821199 (+ 787us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:12:27.821831 (+ 632us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:12:27.821839 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:12:27.821844 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:12:27.822160 (+ 316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:12:27.822809 (+ 649us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:12:27.822818 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:12:27.823251 (+ 433us) client_negotiation.cc:769] Sending connection context
0807 18:12:27.823543 (+ 292us) client_negotiation.cc:240] Negotiation successful
0807 18:12:27.824152 (+ 609us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":746,"thread_start_us":464,"threads_started":1}
I20260807 18:12:27.846797 21881 rpc_server.cc:307] RPC server started. Bound to: 127.18.154.2:39793
I20260807 18:12:27.846873 22069 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.154.2:39793 every 8 connection(s)
I20260807 18:12:27.848439 21881 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126328808105-19048-0/minicluster-data/ts-1/data/info.pb
I20260807 18:12:27.849278 21913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:27.832225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.2:60255 (local address 127.18.154.62:43221)
0807 18:12:27.832414 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0807 18:12:27.832422 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:12:27.833247 (+ 825us) server_negotiation.cc:407] Connection header received
0807 18:12:27.833458 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:12:27.833464 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:12:27.833599 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:12:27.833720 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:12:27.835348 (+ 1628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:27.836524 (+ 1176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:27.838441 (+ 1917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:27.839252 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:27.839412 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:27.841952 (+ 2540us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:12:27.841962 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:12:27.841971 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:12:27.842021 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:12:27.846019 (+ 3998us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:12:27.847074 (+ 1055us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:12:27.847079 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:12:27.847083 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:12:27.847297 (+ 214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:12:27.847662 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:12:27.847670 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:12:27.847674 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:12:27.848094 (+ 420us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:12:27.848260 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0807 18:12:27.848668 (+ 408us) server_negotiation.cc:299] Negotiation successful
0807 18:12:27.848918 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260807 18:12:27.849406 22026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:27.832348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:43221 (local address 127.18.154.2:60255)
0807 18:12:27.833124 (+ 776us) negotiation.cc:107] Waiting for socket to connect
0807 18:12:27.833150 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0807 18:12:27.833331 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:12:27.833782 (+ 451us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:12:27.833788 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:12:27.834635 (+ 847us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:12:27.835172 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:12:27.835184 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:12:27.836680 (+ 1496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:12:27.836684 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:12:27.838283 (+ 1599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:12:27.838296 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:12:27.839494 (+ 1198us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:12:27.839506 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:12:27.839778 (+ 272us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:27.840757 (+ 979us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:12:27.840785 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:12:27.841694 (+ 909us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:12:27.846195 (+ 4501us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:12:27.846207 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:12:27.846213 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:12:27.846889 (+ 676us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:12:27.847402 (+ 513us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:12:27.847407 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:12:27.847410 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:12:27.847547 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:12:27.848220 (+ 673us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:12:27.848225 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:12:27.848420 (+ 195us) client_negotiation.cc:769] Sending connection context
0807 18:12:27.848571 (+ 151us) client_negotiation.cc:240] Negotiation successful
0807 18:12:27.848947 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":622,"thread_start_us":470,"threads_started":1}
I20260807 18:12:27.852725 22071 heartbeater.cc:347] Connected to a master server at 127.18.154.62:43221
I20260807 18:12:27.852994 22071 heartbeater.cc:464] Registering TS with master...
I20260807 18:12:27.853889 22071 heartbeater.cc:511] Master 127.18.154.62:43221 requested a full tablet report, sending...
I20260807 18:12:27.855448 21594 ts_manager.cc:194] Registered new tserver with Master: bf17273192884a1e8a41b74403970a98 (127.18.154.2:39793)
I20260807 18:12:27.856392 21594 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.154.2@KRBTEST.COM'} at 127.18.154.2:52599
I20260807 18:12:27.857815 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 21881
I20260807 18:12:27.858016 19048 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126328808105-19048-0/minicluster-data/ts-1/wal/instance
I20260807 18:12:27.858453 19048 external_mini_cluster.cc:1428] Setting key 5441604677dd95563cb31c080eff4227
WARNING: no policy specified for kudu/127.18.154.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.154.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.154.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.3.keytab.
Entry for principal kudu/127.18.154.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.3.keytab.
I20260807 18:12:27.906734 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126328808105-19048-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126328808105-19048-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126328808105-19048-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126328808105-19048-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.154.3:0
--local_ip_for_outbound_sockets=127.18.154.3
--webserver_interface=127.18.154.3
--webserver_port=0
--tserver_master_addrs=127.18.154.62:43221
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.3.keytab
--principal=kudu/127.18.154.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.154.20:44629
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126328808105-19048-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126328808105-19048-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126328808105-19048-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:12:28.259313 22076 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:12:28.259603 22076 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:12:28.259810 22076 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:12:28.267277 22076 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:12:28.267388 22076 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:12:28.267552 22076 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260807 18:12:28.267632 22076 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 18:12:28.267781 22076 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.154.3
I20260807 18:12:28.280176 22076 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.154.20:44629
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126328808105-19048-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126328808105-19048-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.3.keytab
--principal=kudu/127.18.154.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.154.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126328808105-19048-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.154.3
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.18.154.62:43221
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.154.3
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126328808105-19048-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:12:28.290189 22076 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:12:28.292363 22076 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 18:12:28.296501 21877 heartbeater.cc:503] Master 127.18.154.62:43221 was elected leader, sending a full tablet report...
W20260807 18:12:28.311904 22081 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:12:28.311990 22082 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:12:28.311952 22084 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:12:28.313241 22076 server_base.cc:1034] running on GCE node
I20260807 18:12:28.314088 22076 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:12:28.315130 22076 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:12:28.316428 22076 hybrid_clock.cc:648] HybridClock initialized: now 1786126348316361 us; error 50 us; skew 500 ppm
I20260807 18:12:28.316829 22076 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:12:28 dist-test-slave-gkw7 krb5kdc[21531](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126348, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:12:28.320592 22076 init.cc:377] Logged in from keytab as kudu/127.18.154.3@KRBTEST.COM (short username kudu)
I20260807 18:12:28.322420 22076 webserver.cc:460] Webserver started at http://127.18.154.3:36181/ using document root <none> and password file <none>
I20260807 18:12:28.323239 22076 fs_manager.cc:359] Metadata directory not provided
I20260807 18:12:28.323410 22076 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:12:28.323784 22076 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:12:28.327576 22076 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126328808105-19048-0/minicluster-data/ts-2/data/instance:
uuid: "f5c386f2052b4c86a1c451abcc9ca25e"
format_stamp: "Formatted at 2026-08-07 18:12:28 on dist-test-slave-gkw7"
server_key: "5be4fcc2e4f8a6bb310b4bb322cf5294"
server_key_iv: "64ae5a56c24408b2b2cf3cbbb6829b19"
server_key_version: "encryptionkey@0"
I20260807 18:12:28.328583 22076 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126328808105-19048-0/minicluster-data/ts-2/wal/instance:
uuid: "f5c386f2052b4c86a1c451abcc9ca25e"
format_stamp: "Formatted at 2026-08-07 18:12:28 on dist-test-slave-gkw7"
server_key: "5be4fcc2e4f8a6bb310b4bb322cf5294"
server_key_iv: "64ae5a56c24408b2b2cf3cbbb6829b19"
server_key_version: "encryptionkey@0"
I20260807 18:12:28.335111 22076 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.004s
I20260807 18:12:28.339012 22093 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:12:28.340494 22076 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 18:12:28.340700 22076 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126328808105-19048-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126328808105-19048-0/minicluster-data/ts-2/wal
uuid: "f5c386f2052b4c86a1c451abcc9ca25e"
format_stamp: "Formatted at 2026-08-07 18:12:28 on dist-test-slave-gkw7"
server_key: "5be4fcc2e4f8a6bb310b4bb322cf5294"
server_key_iv: "64ae5a56c24408b2b2cf3cbbb6829b19"
server_key_version: "encryptionkey@0"
I20260807 18:12:28.340945 22076 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126328808105-19048-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126328808105-19048-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126328808105-19048-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:12:28.359201 22076 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:12:28.364001 22076 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:12:28.364334 22076 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:12:28.368386 22076 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:12:28.368530 22076 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:12:28.368721 22076 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:12:28.368866 22076 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 18:12:28 dist-test-slave-gkw7 krb5kdc[21531](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126348, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.3@KRBTEST.COM for kudu/127.18.154.62@KRBTEST.COM
I20260807 18:12:28.394093 22102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:28.368279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.3:41689 (local address 127.18.154.62:43221)
0807 18:12:28.368989 (+ 710us) server_negotiation.cc:206] Beginning negotiation
0807 18:12:28.368998 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:12:28.370198 (+ 1200us) server_negotiation.cc:407] Connection header received
0807 18:12:28.372362 (+ 2164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:12:28.372370 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:12:28.372485 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:12:28.372651 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:12:28.374785 (+ 2134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:28.376414 (+ 1629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:28.380362 (+ 3948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:28.380919 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:28.381093 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:28.386674 (+ 5581us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:12:28.386682 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:12:28.386687 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:12:28.386735 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:12:28.390504 (+ 3769us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:12:28.391554 (+ 1050us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:12:28.391560 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:12:28.391564 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:12:28.391721 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:12:28.392191 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:12:28.392196 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:12:28.392201 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:12:28.392486 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:12:28.392650 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0807 18:12:28.393425 (+ 775us) server_negotiation.cc:299] Negotiation successful
0807 18:12:28.393687 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":567,"thread_start_us":382,"threads_started":1}
I20260807 18:12:28.396184 22104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:28.369003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:43221 (local address 127.18.154.3:41689)
0807 18:12:28.369998 (+ 995us) negotiation.cc:107] Waiting for socket to connect
0807 18:12:28.370057 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0807 18:12:28.372053 (+ 1996us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:12:28.372922 (+ 869us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:12:28.372930 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:12:28.373477 (+ 547us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:12:28.374545 (+ 1068us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:12:28.374581 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:12:28.376685 (+ 2104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:12:28.376696 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:12:28.380139 (+ 3443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:12:28.380156 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:12:28.381305 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:12:28.381316 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:12:28.381822 (+ 506us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:28.382878 (+ 1056us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:12:28.382918 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:12:28.386324 (+ 3406us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:12:28.390723 (+ 4399us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:12:28.390739 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:12:28.390767 (+ 28us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:12:28.391418 (+ 651us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:12:28.391836 (+ 418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:12:28.391841 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:12:28.391845 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:12:28.392061 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:12:28.392642 (+ 581us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:12:28.392650 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:12:28.393038 (+ 388us) client_negotiation.cc:769] Sending connection context
0807 18:12:28.393319 (+ 281us) client_negotiation.cc:240] Negotiation successful
0807 18:12:28.393944 (+ 625us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":719,"thread_start_us":404,"threads_started":1}
I20260807 18:12:28.420498 22102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:28.404112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.3:39553 (local address 127.18.154.62:43221)
0807 18:12:28.404304 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0807 18:12:28.404312 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:12:28.405137 (+ 825us) server_negotiation.cc:407] Connection header received
0807 18:12:28.405484 (+ 347us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:12:28.405490 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:12:28.405637 (+ 147us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:12:28.405750 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:12:28.407499 (+ 1749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:28.408968 (+ 1469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:28.410791 (+ 1823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:28.411335 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:28.411486 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:28.413371 (+ 1885us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:12:28.413377 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:12:28.413383 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:12:28.413422 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:12:28.417143 (+ 3721us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:12:28.418210 (+ 1067us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:12:28.418216 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:12:28.418220 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:12:28.418396 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:12:28.418886 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:12:28.418893 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:12:28.418896 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:12:28.419281 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:12:28.419463 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0807 18:12:28.420060 (+ 597us) server_negotiation.cc:299] Negotiation successful
0807 18:12:28.420253 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260807 18:12:28.421300 22206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:28.404254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:43221 (local address 127.18.154.3:39553)
0807 18:12:28.405000 (+ 746us) negotiation.cc:107] Waiting for socket to connect
0807 18:12:28.405027 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0807 18:12:28.405276 (+ 249us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:12:28.405964 (+ 688us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:12:28.405971 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:12:28.406695 (+ 724us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:12:28.407300 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:12:28.407322 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:12:28.409129 (+ 1807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:12:28.409135 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:12:28.410637 (+ 1502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:12:28.410653 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:12:28.411539 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:12:28.411546 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:12:28.411736 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:28.412413 (+ 677us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:12:28.412431 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:12:28.413174 (+ 743us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:12:28.417311 (+ 4137us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:12:28.417318 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:12:28.417322 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:12:28.418079 (+ 757us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:12:28.418568 (+ 489us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:12:28.418573 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:12:28.418577 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:12:28.418749 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:12:28.419448 (+ 699us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:12:28.419457 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:12:28.419757 (+ 300us) client_negotiation.cc:769] Sending connection context
0807 18:12:28.419992 (+ 235us) client_negotiation.cc:240] Negotiation successful
0807 18:12:28.420549 (+ 557us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":594,"thread_start_us":449,"threads_started":1}
I20260807 18:12:28.428103 22076 rpc_server.cc:307] RPC server started. Bound to: 127.18.154.3:43031
I20260807 18:12:28.428237 22253 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.154.3:43031 every 8 connection(s)
I20260807 18:12:28.429669 22076 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126328808105-19048-0/minicluster-data/ts-2/data/info.pb
I20260807 18:12:28.431609 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 22076
I20260807 18:12:28.431838 19048 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126328808105-19048-0/minicluster-data/ts-2/wal/instance
I20260807 18:12:28.432291 22254 heartbeater.cc:347] Connected to a master server at 127.18.154.62:43221
I20260807 18:12:28.432384 19048 external_mini_cluster.cc:1428] Setting key 71ced6e8ced28c911b21619908e578be
I20260807 18:12:28.432575 22254 heartbeater.cc:464] Registering TS with master...
I20260807 18:12:28.433372 22254 heartbeater.cc:511] Master 127.18.154.62:43221 requested a full tablet report, sending...
I20260807 18:12:28.435129 21593 ts_manager.cc:194] Registered new tserver with Master: f5c386f2052b4c86a1c451abcc9ca25e (127.18.154.3:43031)
I20260807 18:12:28.436233 21593 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.154.3@KRBTEST.COM'} at 127.18.154.3:41689
I20260807 18:12:28.448649 19048 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 18:12:28 dist-test-slave-gkw7 krb5kdc[21531](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126348, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 18:12:28 dist-test-slave-gkw7 krb5kdc[21531](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126348, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.154.62@KRBTEST.COM
I20260807 18:12:28.504618 22102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:28.489014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58836 (local address 127.18.154.62:43221)
0807 18:12:28.489266 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0807 18:12:28.489273 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:12:28.489496 (+ 223us) server_negotiation.cc:407] Connection header received
0807 18:12:28.489780 (+ 284us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:12:28.489785 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:12:28.489854 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:12:28.489973 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:12:28.491461 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:28.492229 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:28.493512 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:28.493975 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:28.494120 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:28.498601 (+ 4481us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:12:28.498606 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:12:28.498610 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:12:28.498649 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:12:28.501958 (+ 3309us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:12:28.502851 (+ 893us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:12:28.502858 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:12:28.502862 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:12:28.503029 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:12:28.503401 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:12:28.503405 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:12:28.503407 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:12:28.503707 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:12:28.503913 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0807 18:12:28.504180 (+ 267us) server_negotiation.cc:299] Negotiation successful
0807 18:12:28.504354 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260807 18:12:28.509382 21593 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58836:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 18:12:28.516569 21593 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:12:28.553867 22272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:28.529158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.1:38089 (local address 127.0.0.1:55412)
0807 18:12:28.542857 (+ 13699us) negotiation.cc:107] Waiting for socket to connect
0807 18:12:28.542910 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0807 18:12:28.543281 (+ 371us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:12:28.544442 (+ 1161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:12:28.544452 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:12:28.544576 (+ 124us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:12:28.545099 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:12:28.545115 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:12:28.547573 (+ 2458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:12:28.547581 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:12:28.550520 (+ 2939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:12:28.550537 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:12:28.552836 (+ 2299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:12:28.552842 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:12:28.553060 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:28.553115 (+ 55us) client_negotiation.cc:769] Sending connection context
0807 18:12:28.553234 (+ 119us) client_negotiation.cc:240] Negotiation successful
0807 18:12:28.553416 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13360,"thread_start_us":13213,"threads_started":1}
I20260807 18:12:28.554136 22273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:28.529451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55412 (local address 127.18.154.1:38089)
0807 18:12:28.541948 (+ 12497us) server_negotiation.cc:206] Beginning negotiation
0807 18:12:28.541963 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 18:12:28.543073 (+ 1110us) server_negotiation.cc:407] Connection header received
0807 18:12:28.543591 (+ 518us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:12:28.543604 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:12:28.543924 (+ 320us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:12:28.544119 (+ 195us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:12:28.545316 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:28.547273 (+ 1957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:28.550738 (+ 3465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:28.552332 (+ 1594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:28.552464 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:28.552694 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0807 18:12:28.553677 (+ 983us) server_negotiation.cc:299] Negotiation successful
0807 18:12:28.553854 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12229,"thread_start_us":12493,"threads_started":1}
I20260807 18:12:28.559088 22274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:28.529982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.2:39793 (local address 127.0.0.1:36944)
0807 18:12:28.549705 (+ 19723us) negotiation.cc:107] Waiting for socket to connect
0807 18:12:28.549730 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0807 18:12:28.549889 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:12:28.550750 (+ 861us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:12:28.550754 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:12:28.550783 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:12:28.551167 (+ 384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:12:28.551179 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:12:28.553651 (+ 2472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:12:28.553658 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:12:28.556273 (+ 2615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:12:28.556289 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:12:28.558210 (+ 1921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:12:28.558217 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:12:28.558387 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:28.558426 (+ 39us) client_negotiation.cc:769] Sending connection context
0807 18:12:28.558531 (+ 105us) client_negotiation.cc:240] Negotiation successful
0807 18:12:28.558689 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19553,"mutex_wait_us":252,"thread_start_us":19387,"threads_started":1}
I20260807 18:12:28.559113 22276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:28.530143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36944 (local address 127.18.154.2:39793)
0807 18:12:28.548900 (+ 18757us) server_negotiation.cc:206] Beginning negotiation
0807 18:12:28.548914 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 18:12:28.550053 (+ 1139us) server_negotiation.cc:407] Connection header received
0807 18:12:28.550173 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:12:28.550183 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:12:28.550509 (+ 326us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:12:28.550731 (+ 222us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:12:28.551363 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:28.553432 (+ 2069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:28.556485 (+ 3053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:28.557961 (+ 1476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:28.558120 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:28.558364 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0807 18:12:28.558605 (+ 241us) server_negotiation.cc:299] Negotiation successful
0807 18:12:28.558795 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18495,"thread_start_us":18738,"threads_started":1}
I20260807 18:12:28.558585 21784 tablet_service.cc:1467] Processing CreateTablet for tablet 0605f98bc83e477ca5ed7f98effe4981 (DEFAULT_TABLE table=test-table [id=cae4ad4aadb64a7780008ac6b6fbd4ed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:12:28.559701 22277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:28.530352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.3:43031 (local address 127.0.0.1:45078)
0807 18:12:28.550074 (+ 19722us) negotiation.cc:107] Waiting for socket to connect
0807 18:12:28.550093 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 18:12:28.550193 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:12:28.551331 (+ 1138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:12:28.551336 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:12:28.551362 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:12:28.551786 (+ 424us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:12:28.551800 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:12:28.553612 (+ 1812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:12:28.553618 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:12:28.556271 (+ 2653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:12:28.556289 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:12:28.558740 (+ 2451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:12:28.558747 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:12:28.558916 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:28.558954 (+ 38us) client_negotiation.cc:769] Sending connection context
0807 18:12:28.559054 (+ 100us) client_negotiation.cc:240] Negotiation successful
0807 18:12:28.559195 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19604,"mutex_wait_us":51,"thread_start_us":19440,"threads_started":1}
I20260807 18:12:28.560359 21784 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0605f98bc83e477ca5ed7f98effe4981. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:12:28.560518 22275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:28.529977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45078 (local address 127.18.154.3:43031)
0807 18:12:28.550837 (+ 20860us) server_negotiation.cc:206] Beginning negotiation
0807 18:12:28.550846 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:12:28.550880 (+ 34us) server_negotiation.cc:407] Connection header received
0807 18:12:28.550975 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:12:28.550980 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:12:28.551128 (+ 148us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:12:28.551349 (+ 221us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:12:28.552026 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:28.553440 (+ 1414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:28.556678 (+ 3238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:28.558193 (+ 1515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:28.558316 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:28.558578 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0807 18:12:28.560065 (+ 1487us) server_negotiation.cc:299] Negotiation successful
0807 18:12:28.560192 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20666,"thread_start_us":20465,"threads_started":1}
I20260807 18:12:28.563172 21970 tablet_service.cc:1467] Processing CreateTablet for tablet 0605f98bc83e477ca5ed7f98effe4981 (DEFAULT_TABLE table=test-table [id=cae4ad4aadb64a7780008ac6b6fbd4ed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:12:28.563859 22163 tablet_service.cc:1467] Processing CreateTablet for tablet 0605f98bc83e477ca5ed7f98effe4981 (DEFAULT_TABLE table=test-table [id=cae4ad4aadb64a7780008ac6b6fbd4ed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:12:28.564450 21970 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0605f98bc83e477ca5ed7f98effe4981. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:12:28.565201 22163 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0605f98bc83e477ca5ed7f98effe4981. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:12:28.576956 22278 tablet_bootstrap.cc:492] T 0605f98bc83e477ca5ed7f98effe4981 P f5c386f2052b4c86a1c451abcc9ca25e: Bootstrap starting.
I20260807 18:12:28.579372 22280 tablet_bootstrap.cc:492] T 0605f98bc83e477ca5ed7f98effe4981 P bf17273192884a1e8a41b74403970a98: Bootstrap starting.
I20260807 18:12:28.580191 22279 tablet_bootstrap.cc:492] T 0605f98bc83e477ca5ed7f98effe4981 P f7ddbaeafa194da88d61994cf63503f6: Bootstrap starting.
I20260807 18:12:28.580919 22278 tablet_bootstrap.cc:654] T 0605f98bc83e477ca5ed7f98effe4981 P f5c386f2052b4c86a1c451abcc9ca25e: Neither blocks nor log segments found. Creating new log.
I20260807 18:12:28.582381 22278 log.cc:824] T 0605f98bc83e477ca5ed7f98effe4981 P f5c386f2052b4c86a1c451abcc9ca25e: Log is configured to *not* fsync() on all Append() calls
I20260807 18:12:28.584082 22279 tablet_bootstrap.cc:654] T 0605f98bc83e477ca5ed7f98effe4981 P f7ddbaeafa194da88d61994cf63503f6: Neither blocks nor log segments found. Creating new log.
I20260807 18:12:28.584604 22280 tablet_bootstrap.cc:654] T 0605f98bc83e477ca5ed7f98effe4981 P bf17273192884a1e8a41b74403970a98: Neither blocks nor log segments found. Creating new log.
I20260807 18:12:28.585506 22279 log.cc:824] T 0605f98bc83e477ca5ed7f98effe4981 P f7ddbaeafa194da88d61994cf63503f6: Log is configured to *not* fsync() on all Append() calls
I20260807 18:12:28.585830 22280 log.cc:824] T 0605f98bc83e477ca5ed7f98effe4981 P bf17273192884a1e8a41b74403970a98: Log is configured to *not* fsync() on all Append() calls
I20260807 18:12:28.586192 22278 tablet_bootstrap.cc:492] T 0605f98bc83e477ca5ed7f98effe4981 P f5c386f2052b4c86a1c451abcc9ca25e: No bootstrap required, opened a new log
I20260807 18:12:28.586570 22278 ts_tablet_manager.cc:1397] T 0605f98bc83e477ca5ed7f98effe4981 P f5c386f2052b4c86a1c451abcc9ca25e: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.002s
I20260807 18:12:28.588881 22280 tablet_bootstrap.cc:492] T 0605f98bc83e477ca5ed7f98effe4981 P bf17273192884a1e8a41b74403970a98: No bootstrap required, opened a new log
I20260807 18:12:28.588876 22279 tablet_bootstrap.cc:492] T 0605f98bc83e477ca5ed7f98effe4981 P f7ddbaeafa194da88d61994cf63503f6: No bootstrap required, opened a new log
I20260807 18:12:28.589169 22279 ts_tablet_manager.cc:1397] T 0605f98bc83e477ca5ed7f98effe4981 P f7ddbaeafa194da88d61994cf63503f6: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.003s
I20260807 18:12:28.589169 22280 ts_tablet_manager.cc:1397] T 0605f98bc83e477ca5ed7f98effe4981 P bf17273192884a1e8a41b74403970a98: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.000s
I20260807 18:12:28.594173 22278 raft_consensus.cc:348] T 0605f98bc83e477ca5ed7f98effe4981 P f5c386f2052b4c86a1c451abcc9ca25e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7ddbaeafa194da88d61994cf63503f6" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 38089 } } peers { permanent_uuid: "f5c386f2052b4c86a1c451abcc9ca25e" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 43031 } } peers { permanent_uuid: "bf17273192884a1e8a41b74403970a98" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 39793 } }
I20260807 18:12:28.594786 22278 raft_consensus.cc:374] T 0605f98bc83e477ca5ed7f98effe4981 P f5c386f2052b4c86a1c451abcc9ca25e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:12:28.594950 22278 raft_consensus.cc:729] T 0605f98bc83e477ca5ed7f98effe4981 P f5c386f2052b4c86a1c451abcc9ca25e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5c386f2052b4c86a1c451abcc9ca25e, State: Initialized, Role: FOLLOWER
I20260807 18:12:28.595726 22278 consensus_queue.cc:260] T 0605f98bc83e477ca5ed7f98effe4981 P f5c386f2052b4c86a1c451abcc9ca25e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7ddbaeafa194da88d61994cf63503f6" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 38089 } } peers { permanent_uuid: "f5c386f2052b4c86a1c451abcc9ca25e" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 43031 } } peers { permanent_uuid: "bf17273192884a1e8a41b74403970a98" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 39793 } }
I20260807 18:12:28.596926 22254 heartbeater.cc:503] Master 127.18.154.62:43221 was elected leader, sending a full tablet report...
I20260807 18:12:28.597918 22278 ts_tablet_manager.cc:1428] T 0605f98bc83e477ca5ed7f98effe4981 P f5c386f2052b4c86a1c451abcc9ca25e: Time spent starting tablet: real 0.011s user 0.005s sys 0.006s
I20260807 18:12:28.600284 22279 raft_consensus.cc:348] T 0605f98bc83e477ca5ed7f98effe4981 P f7ddbaeafa194da88d61994cf63503f6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7ddbaeafa194da88d61994cf63503f6" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 38089 } } peers { permanent_uuid: "f5c386f2052b4c86a1c451abcc9ca25e" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 43031 } } peers { permanent_uuid: "bf17273192884a1e8a41b74403970a98" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 39793 } }
I20260807 18:12:28.600757 22279 raft_consensus.cc:374] T 0605f98bc83e477ca5ed7f98effe4981 P f7ddbaeafa194da88d61994cf63503f6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:12:28.600867 22279 raft_consensus.cc:729] T 0605f98bc83e477ca5ed7f98effe4981 P f7ddbaeafa194da88d61994cf63503f6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7ddbaeafa194da88d61994cf63503f6, State: Initialized, Role: FOLLOWER
I20260807 18:12:28.600646 22280 raft_consensus.cc:348] T 0605f98bc83e477ca5ed7f98effe4981 P bf17273192884a1e8a41b74403970a98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7ddbaeafa194da88d61994cf63503f6" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 38089 } } peers { permanent_uuid: "f5c386f2052b4c86a1c451abcc9ca25e" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 43031 } } peers { permanent_uuid: "bf17273192884a1e8a41b74403970a98" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 39793 } }
I20260807 18:12:28.601182 22280 raft_consensus.cc:374] T 0605f98bc83e477ca5ed7f98effe4981 P bf17273192884a1e8a41b74403970a98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:12:28.601330 22280 raft_consensus.cc:729] T 0605f98bc83e477ca5ed7f98effe4981 P bf17273192884a1e8a41b74403970a98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf17273192884a1e8a41b74403970a98, State: Initialized, Role: FOLLOWER
I20260807 18:12:28.601505 22279 consensus_queue.cc:260] T 0605f98bc83e477ca5ed7f98effe4981 P f7ddbaeafa194da88d61994cf63503f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7ddbaeafa194da88d61994cf63503f6" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 38089 } } peers { permanent_uuid: "f5c386f2052b4c86a1c451abcc9ca25e" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 43031 } } peers { permanent_uuid: "bf17273192884a1e8a41b74403970a98" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 39793 } }
I20260807 18:12:28.602075 22280 consensus_queue.cc:260] T 0605f98bc83e477ca5ed7f98effe4981 P bf17273192884a1e8a41b74403970a98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7ddbaeafa194da88d61994cf63503f6" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 38089 } } peers { permanent_uuid: "f5c386f2052b4c86a1c451abcc9ca25e" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 43031 } } peers { permanent_uuid: "bf17273192884a1e8a41b74403970a98" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 39793 } }
I20260807 18:12:28.603646 22279 ts_tablet_manager.cc:1428] T 0605f98bc83e477ca5ed7f98effe4981 P f7ddbaeafa194da88d61994cf63503f6: Time spent starting tablet: real 0.014s user 0.011s sys 0.004s
I20260807 18:12:28.604705 22071 heartbeater.cc:503] Master 127.18.154.62:43221 was elected leader, sending a full tablet report...
I20260807 18:12:28.605094 22280 ts_tablet_manager.cc:1428] T 0605f98bc83e477ca5ed7f98effe4981 P bf17273192884a1e8a41b74403970a98: Time spent starting tablet: real 0.016s user 0.012s sys 0.004s
W20260807 18:12:28.681788 22258 tablet.cc:2369] T 0605f98bc83e477ca5ed7f98effe4981 P f5c386f2052b4c86a1c451abcc9ca25e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 18:12:28.788435 21878 tablet.cc:2369] T 0605f98bc83e477ca5ed7f98effe4981 P f7ddbaeafa194da88d61994cf63503f6: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 18:12:28.852310 22073 tablet.cc:2369] T 0605f98bc83e477ca5ed7f98effe4981 P bf17273192884a1e8a41b74403970a98: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:12:28.997383 22286 raft_consensus.cc:482] T 0605f98bc83e477ca5ed7f98effe4981 P f7ddbaeafa194da88d61994cf63503f6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:12:28.997792 22286 raft_consensus.cc:504] T 0605f98bc83e477ca5ed7f98effe4981 P f7ddbaeafa194da88d61994cf63503f6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7ddbaeafa194da88d61994cf63503f6" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 38089 } } peers { permanent_uuid: "f5c386f2052b4c86a1c451abcc9ca25e" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 43031 } } peers { permanent_uuid: "bf17273192884a1e8a41b74403970a98" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 39793 } }
I20260807 18:12:29.001401 22286 leader_election.cc:288] T 0605f98bc83e477ca5ed7f98effe4981 P f7ddbaeafa194da88d61994cf63503f6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f5c386f2052b4c86a1c451abcc9ca25e (127.18.154.3:43031), bf17273192884a1e8a41b74403970a98 (127.18.154.2:39793)
I20260807 18:12:29.009197 22275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:29.000618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.1:48423 (local address 127.18.154.3:43031)
0807 18:12:29.000866 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0807 18:12:29.000875 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:12:29.001606 (+ 731us) server_negotiation.cc:407] Connection header received
0807 18:12:29.001927 (+ 321us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:12:29.001936 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:12:29.002049 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:12:29.002179 (+ 130us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:12:29.002687 (+ 508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:29.004155 (+ 1468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:29.006235 (+ 2080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:29.007962 (+ 1727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:29.008201 (+ 239us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:29.008308 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0807 18:12:29.008764 (+ 456us) server_negotiation.cc:299] Negotiation successful
0807 18:12:29.008938 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260807 18:12:29.009212 22289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:29.000416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.3:43031 (local address 127.18.154.1:48423)
0807 18:12:29.001464 (+ 1048us) negotiation.cc:107] Waiting for socket to connect
0807 18:12:29.001484 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 18:12:29.001733 (+ 249us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:12:29.002187 (+ 454us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:12:29.002190 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:12:29.002210 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:12:29.002535 (+ 325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:12:29.002543 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:12:29.004293 (+ 1750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:12:29.004297 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:12:29.006017 (+ 1720us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:12:29.006030 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:12:29.008270 (+ 2240us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:12:29.008284 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:12:29.008548 (+ 264us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:29.008593 (+ 45us) client_negotiation.cc:769] Sending connection context
0807 18:12:29.008722 (+ 129us) client_negotiation.cc:240] Negotiation successful
0807 18:12:29.008868 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":882,"thread_start_us":7963,"threads_started":1}
I20260807 18:12:29.010239 22187 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0605f98bc83e477ca5ed7f98effe4981" candidate_uuid: "f7ddbaeafa194da88d61994cf63503f6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5c386f2052b4c86a1c451abcc9ca25e" is_pre_election: true
I20260807 18:12:29.010725 22187 raft_consensus.cc:2454] T 0605f98bc83e477ca5ed7f98effe4981 P f5c386f2052b4c86a1c451abcc9ca25e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f7ddbaeafa194da88d61994cf63503f6 in term 0.
I20260807 18:12:29.011425 21717 leader_election.cc:302] T 0605f98bc83e477ca5ed7f98effe4981 P f7ddbaeafa194da88d61994cf63503f6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f5c386f2052b4c86a1c451abcc9ca25e, f7ddbaeafa194da88d61994cf63503f6; no voters:
I20260807 18:12:29.011866 22286 raft_consensus.cc:2781] T 0605f98bc83e477ca5ed7f98effe4981 P f7ddbaeafa194da88d61994cf63503f6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:12:29.012013 22286 raft_consensus.cc:482] T 0605f98bc83e477ca5ed7f98effe4981 P f7ddbaeafa194da88d61994cf63503f6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:12:29.012156 22286 raft_consensus.cc:3037] T 0605f98bc83e477ca5ed7f98effe4981 P f7ddbaeafa194da88d61994cf63503f6 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:12:29.014680 22288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:29.000637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.2:39793 (local address 127.18.154.1:41333)
0807 18:12:29.008718 (+ 8081us) negotiation.cc:107] Waiting for socket to connect
0807 18:12:29.008738 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 18:12:29.008865 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:12:29.009223 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:12:29.009231 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:12:29.009263 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:12:29.009722 (+ 459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:12:29.009735 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:12:29.010925 (+ 1190us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:12:29.010930 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:12:29.012506 (+ 1576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:12:29.012519 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:12:29.014106 (+ 1587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:12:29.014111 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:12:29.014236 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:29.014260 (+ 24us) client_negotiation.cc:769] Sending connection context
0807 18:12:29.014335 (+ 75us) client_negotiation.cc:240] Negotiation successful
0807 18:12:29.014429 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7934,"thread_start_us":450,"threads_started":1}
I20260807 18:12:29.014798 22276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:29.000599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.1:41333 (local address 127.18.154.2:39793)
0807 18:12:29.000837 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0807 18:12:29.000845 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:12:29.008851 (+ 8006us) server_negotiation.cc:407] Connection header received
0807 18:12:29.008981 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:12:29.008986 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:12:29.009071 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:12:29.009170 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:12:29.009939 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:29.010764 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:29.012668 (+ 1904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:29.013901 (+ 1233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:29.014071 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:29.014147 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0807 18:12:29.014401 (+ 254us) server_negotiation.cc:299] Negotiation successful
0807 18:12:29.014545 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260807 18:12:29.015588 22286 raft_consensus.cc:504] T 0605f98bc83e477ca5ed7f98effe4981 P f7ddbaeafa194da88d61994cf63503f6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7ddbaeafa194da88d61994cf63503f6" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 38089 } } peers { permanent_uuid: "f5c386f2052b4c86a1c451abcc9ca25e" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 43031 } } peers { permanent_uuid: "bf17273192884a1e8a41b74403970a98" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 39793 } }
I20260807 18:12:29.015911 22000 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0605f98bc83e477ca5ed7f98effe4981" candidate_uuid: "f7ddbaeafa194da88d61994cf63503f6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf17273192884a1e8a41b74403970a98" is_pre_election: true
I20260807 18:12:29.016445 22286 leader_election.cc:288] T 0605f98bc83e477ca5ed7f98effe4981 P f7ddbaeafa194da88d61994cf63503f6 [CANDIDATE]: Term 1 election: Requested vote from peers f5c386f2052b4c86a1c451abcc9ca25e (127.18.154.3:43031), bf17273192884a1e8a41b74403970a98 (127.18.154.2:39793)
I20260807 18:12:29.016587 22000 raft_consensus.cc:2454] T 0605f98bc83e477ca5ed7f98effe4981 P bf17273192884a1e8a41b74403970a98 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f7ddbaeafa194da88d61994cf63503f6 in term 0.
I20260807 18:12:29.016956 21999 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0605f98bc83e477ca5ed7f98effe4981" candidate_uuid: "f7ddbaeafa194da88d61994cf63503f6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf17273192884a1e8a41b74403970a98"
I20260807 18:12:29.016978 22187 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0605f98bc83e477ca5ed7f98effe4981" candidate_uuid: "f7ddbaeafa194da88d61994cf63503f6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5c386f2052b4c86a1c451abcc9ca25e"
I20260807 18:12:29.017239 21999 raft_consensus.cc:3037] T 0605f98bc83e477ca5ed7f98effe4981 P bf17273192884a1e8a41b74403970a98 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:12:29.017218 22187 raft_consensus.cc:3037] T 0605f98bc83e477ca5ed7f98effe4981 P f5c386f2052b4c86a1c451abcc9ca25e [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:12:29.020116 22187 raft_consensus.cc:2454] T 0605f98bc83e477ca5ed7f98effe4981 P f5c386f2052b4c86a1c451abcc9ca25e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f7ddbaeafa194da88d61994cf63503f6 in term 1.
I20260807 18:12:29.020182 21999 raft_consensus.cc:2454] T 0605f98bc83e477ca5ed7f98effe4981 P bf17273192884a1e8a41b74403970a98 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f7ddbaeafa194da88d61994cf63503f6 in term 1.
I20260807 18:12:29.020550 21717 leader_election.cc:302] T 0605f98bc83e477ca5ed7f98effe4981 P f7ddbaeafa194da88d61994cf63503f6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f5c386f2052b4c86a1c451abcc9ca25e, f7ddbaeafa194da88d61994cf63503f6; no voters:
I20260807 18:12:29.020874 22286 raft_consensus.cc:2781] T 0605f98bc83e477ca5ed7f98effe4981 P f7ddbaeafa194da88d61994cf63503f6 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:12:29.021356 22286 raft_consensus.cc:686] T 0605f98bc83e477ca5ed7f98effe4981 P f7ddbaeafa194da88d61994cf63503f6 [term 1 LEADER]: Becoming Leader. State: Replica: f7ddbaeafa194da88d61994cf63503f6, State: Running, Role: LEADER
I20260807 18:12:29.021914 22286 consensus_queue.cc:237] T 0605f98bc83e477ca5ed7f98effe4981 P f7ddbaeafa194da88d61994cf63503f6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7ddbaeafa194da88d61994cf63503f6" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 38089 } } peers { permanent_uuid: "f5c386f2052b4c86a1c451abcc9ca25e" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 43031 } } peers { permanent_uuid: "bf17273192884a1e8a41b74403970a98" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 39793 } }
I20260807 18:12:29.027242 21593 catalog_manager.cc:5495] T 0605f98bc83e477ca5ed7f98effe4981 P f7ddbaeafa194da88d61994cf63503f6 reported cstate change: term changed from 0 to 1, leader changed from <none> to f7ddbaeafa194da88d61994cf63503f6 (127.18.154.1). New cstate: current_term: 1 leader_uuid: "f7ddbaeafa194da88d61994cf63503f6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7ddbaeafa194da88d61994cf63503f6" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 38089 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5c386f2052b4c86a1c451abcc9ca25e" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 43031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bf17273192884a1e8a41b74403970a98" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 39793 } health_report { overall_health: UNKNOWN } } }
Aug 07 18:12:29 dist-test-slave-gkw7 krb5kdc[21531](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126346, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.62@KRBTEST.COM for kudu/127.18.154.62@KRBTEST.COM
I20260807 18:12:29.150274 22310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:29.136177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:43221 (local address 127.0.0.1:58842)
0807 18:12:29.136999 (+ 822us) negotiation.cc:107] Waiting for socket to connect
0807 18:12:29.137017 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 18:12:29.137155 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:12:29.137496 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:12:29.137500 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:12:29.138231 (+ 731us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:12:29.138439 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:12:29.138448 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:12:29.139510 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:12:29.139514 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:12:29.140487 (+ 973us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:12:29.140498 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:12:29.141164 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:12:29.141169 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:12:29.141296 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:29.142100 (+ 804us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:12:29.142118 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:12:29.144473 (+ 2355us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:12:29.147718 (+ 3245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:12:29.147734 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:12:29.147746 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:12:29.148218 (+ 472us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:12:29.148615 (+ 397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:12:29.148619 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:12:29.148621 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:12:29.148784 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:12:29.149276 (+ 492us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:12:29.149282 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:12:29.149441 (+ 159us) client_negotiation.cc:769] Sending connection context
0807 18:12:29.149668 (+ 227us) client_negotiation.cc:240] Negotiation successful
0807 18:12:29.149939 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":576,"thread_start_us":378,"threads_started":1}
I20260807 18:12:29.150370 22307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:29.136207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58842 (local address 127.18.154.62:43221)
0807 18:12:29.137126 (+ 919us) server_negotiation.cc:206] Beginning negotiation
0807 18:12:29.137130 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:12:29.137153 (+ 23us) server_negotiation.cc:407] Connection header received
0807 18:12:29.137287 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:12:29.137291 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:12:29.137350 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:12:29.137443 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:12:29.138593 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:29.139379 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:29.140608 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:29.141024 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:29.141114 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:29.144670 (+ 3556us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:12:29.144675 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:12:29.144678 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:12:29.144708 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:12:29.147557 (+ 2849us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:12:29.148356 (+ 799us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:12:29.148368 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:12:29.148371 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:12:29.148506 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:12:29.148882 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:12:29.148886 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:12:29.148888 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:12:29.149151 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:12:29.149282 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0807 18:12:29.149782 (+ 500us) server_negotiation.cc:299] Negotiation successful
0807 18:12:29.150072 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":103,"server-negotiator.queue_time_us":816,"thread_start_us":668,"threads_started":1}
I20260807 18:12:29.155552 21593 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.18.154.62@KRBTEST.COM'} at 127.0.0.1:58842:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260807 18:12:29.156622 21593 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:12:29.163231 21784 tablet_service.cc:1467] Processing CreateTablet for tablet 989bf36191dd43e490148cf0b546f529 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=559bf951ff314813945681856b382dac]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 18:12:29.163420 22163 tablet_service.cc:1467] Processing CreateTablet for tablet 989bf36191dd43e490148cf0b546f529 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=559bf951ff314813945681856b382dac]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 18:12:29.163273 21970 tablet_service.cc:1467] Processing CreateTablet for tablet 989bf36191dd43e490148cf0b546f529 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=559bf951ff314813945681856b382dac]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 18:12:29.164242 21784 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 989bf36191dd43e490148cf0b546f529. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:12:29.164255 22163 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 989bf36191dd43e490148cf0b546f529. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:12:29.164271 21970 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 989bf36191dd43e490148cf0b546f529. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:12:29.170802 22312 tablet_bootstrap.cc:492] T 989bf36191dd43e490148cf0b546f529 P f7ddbaeafa194da88d61994cf63503f6: Bootstrap starting.
I20260807 18:12:29.172056 22315 tablet_bootstrap.cc:492] T 989bf36191dd43e490148cf0b546f529 P f5c386f2052b4c86a1c451abcc9ca25e: Bootstrap starting.
I20260807 18:12:29.172911 22313 tablet_bootstrap.cc:492] T 989bf36191dd43e490148cf0b546f529 P bf17273192884a1e8a41b74403970a98: Bootstrap starting.
I20260807 18:12:29.173789 22312 tablet_bootstrap.cc:654] T 989bf36191dd43e490148cf0b546f529 P f7ddbaeafa194da88d61994cf63503f6: Neither blocks nor log segments found. Creating new log.
I20260807 18:12:29.173880 22315 tablet_bootstrap.cc:654] T 989bf36191dd43e490148cf0b546f529 P f5c386f2052b4c86a1c451abcc9ca25e: Neither blocks nor log segments found. Creating new log.
I20260807 18:12:29.174464 22313 tablet_bootstrap.cc:654] T 989bf36191dd43e490148cf0b546f529 P bf17273192884a1e8a41b74403970a98: Neither blocks nor log segments found. Creating new log.
I20260807 18:12:29.177487 22313 tablet_bootstrap.cc:492] T 989bf36191dd43e490148cf0b546f529 P bf17273192884a1e8a41b74403970a98: No bootstrap required, opened a new log
I20260807 18:12:29.177690 22313 ts_tablet_manager.cc:1397] T 989bf36191dd43e490148cf0b546f529 P bf17273192884a1e8a41b74403970a98: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s
I20260807 18:12:29.178066 22312 tablet_bootstrap.cc:492] T 989bf36191dd43e490148cf0b546f529 P f7ddbaeafa194da88d61994cf63503f6: No bootstrap required, opened a new log
I20260807 18:12:29.178223 22312 ts_tablet_manager.cc:1397] T 989bf36191dd43e490148cf0b546f529 P f7ddbaeafa194da88d61994cf63503f6: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.003s
I20260807 18:12:29.178483 22313 raft_consensus.cc:348] T 989bf36191dd43e490148cf0b546f529 P bf17273192884a1e8a41b74403970a98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7ddbaeafa194da88d61994cf63503f6" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 38089 } } peers { permanent_uuid: "bf17273192884a1e8a41b74403970a98" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 39793 } } peers { permanent_uuid: "f5c386f2052b4c86a1c451abcc9ca25e" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 43031 } }
I20260807 18:12:29.178725 22313 raft_consensus.cc:374] T 989bf36191dd43e490148cf0b546f529 P bf17273192884a1e8a41b74403970a98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:12:29.178843 22313 raft_consensus.cc:729] T 989bf36191dd43e490148cf0b546f529 P bf17273192884a1e8a41b74403970a98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf17273192884a1e8a41b74403970a98, State: Initialized, Role: FOLLOWER
I20260807 18:12:29.178966 22312 raft_consensus.cc:348] T 989bf36191dd43e490148cf0b546f529 P f7ddbaeafa194da88d61994cf63503f6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7ddbaeafa194da88d61994cf63503f6" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 38089 } } peers { permanent_uuid: "bf17273192884a1e8a41b74403970a98" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 39793 } } peers { permanent_uuid: "f5c386f2052b4c86a1c451abcc9ca25e" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 43031 } }
I20260807 18:12:29.179174 22312 raft_consensus.cc:374] T 989bf36191dd43e490148cf0b546f529 P f7ddbaeafa194da88d61994cf63503f6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:12:29.179105 22313 consensus_queue.cc:260] T 989bf36191dd43e490148cf0b546f529 P bf17273192884a1e8a41b74403970a98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7ddbaeafa194da88d61994cf63503f6" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 38089 } } peers { permanent_uuid: "bf17273192884a1e8a41b74403970a98" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 39793 } } peers { permanent_uuid: "f5c386f2052b4c86a1c451abcc9ca25e" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 43031 } }
I20260807 18:12:29.179311 22312 raft_consensus.cc:729] T 989bf36191dd43e490148cf0b546f529 P f7ddbaeafa194da88d61994cf63503f6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7ddbaeafa194da88d61994cf63503f6, State: Initialized, Role: FOLLOWER
I20260807 18:12:29.179541 22312 consensus_queue.cc:260] T 989bf36191dd43e490148cf0b546f529 P f7ddbaeafa194da88d61994cf63503f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7ddbaeafa194da88d61994cf63503f6" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 38089 } } peers { permanent_uuid: "bf17273192884a1e8a41b74403970a98" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 39793 } } peers { permanent_uuid: "f5c386f2052b4c86a1c451abcc9ca25e" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 43031 } }
I20260807 18:12:29.180002 22313 ts_tablet_manager.cc:1428] T 989bf36191dd43e490148cf0b546f529 P bf17273192884a1e8a41b74403970a98: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260807 18:12:29.180007 22286 tablet_replica.cc:439] T 989bf36191dd43e490148cf0b546f529 P f7ddbaeafa194da88d61994cf63503f6: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7ddbaeafa194da88d61994cf63503f6" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 38089 } } peers { permanent_uuid: "bf17273192884a1e8a41b74403970a98" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 39793 } } peers { permanent_uuid: "f5c386f2052b4c86a1c451abcc9ca25e" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 43031 } } }
I20260807 18:12:29.180317 22286 tablet_replica.cc:442] T 989bf36191dd43e490148cf0b546f529 P f7ddbaeafa194da88d61994cf63503f6: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 18:12:29.180775 22312 ts_tablet_manager.cc:1428] T 989bf36191dd43e490148cf0b546f529 P f7ddbaeafa194da88d61994cf63503f6: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 18:12:29.181016 22315 tablet_bootstrap.cc:492] T 989bf36191dd43e490148cf0b546f529 P f5c386f2052b4c86a1c451abcc9ca25e: No bootstrap required, opened a new log
I20260807 18:12:29.181185 22315 ts_tablet_manager.cc:1397] T 989bf36191dd43e490148cf0b546f529 P f5c386f2052b4c86a1c451abcc9ca25e: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.000s
I20260807 18:12:29.181964 22315 raft_consensus.cc:348] T 989bf36191dd43e490148cf0b546f529 P f5c386f2052b4c86a1c451abcc9ca25e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7ddbaeafa194da88d61994cf63503f6" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 38089 } } peers { permanent_uuid: "bf17273192884a1e8a41b74403970a98" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 39793 } } peers { permanent_uuid: "f5c386f2052b4c86a1c451abcc9ca25e" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 43031 } }
I20260807 18:12:29.182257 22315 raft_consensus.cc:374] T 989bf36191dd43e490148cf0b546f529 P f5c386f2052b4c86a1c451abcc9ca25e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:12:29.182109 22321 tablet_replica.cc:439] T 989bf36191dd43e490148cf0b546f529 P bf17273192884a1e8a41b74403970a98: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7ddbaeafa194da88d61994cf63503f6" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 38089 } } peers { permanent_uuid: "bf17273192884a1e8a41b74403970a98" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 39793 } } peers { permanent_uuid: "f5c386f2052b4c86a1c451abcc9ca25e" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 43031 } } }
I20260807 18:12:29.182387 22315 raft_consensus.cc:729] T 989bf36191dd43e490148cf0b546f529 P f5c386f2052b4c86a1c451abcc9ca25e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5c386f2052b4c86a1c451abcc9ca25e, State: Initialized, Role: FOLLOWER
I20260807 18:12:29.182586 22321 tablet_replica.cc:442] T 989bf36191dd43e490148cf0b546f529 P bf17273192884a1e8a41b74403970a98: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 18:12:29.182621 22315 consensus_queue.cc:260] T 989bf36191dd43e490148cf0b546f529 P f5c386f2052b4c86a1c451abcc9ca25e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7ddbaeafa194da88d61994cf63503f6" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 38089 } } peers { permanent_uuid: "bf17273192884a1e8a41b74403970a98" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 39793 } } peers { permanent_uuid: "f5c386f2052b4c86a1c451abcc9ca25e" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 43031 } }
I20260807 18:12:29.183964 22315 ts_tablet_manager.cc:1428] T 989bf36191dd43e490148cf0b546f529 P f5c386f2052b4c86a1c451abcc9ca25e: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 18:12:29.183995 22322 tablet_replica.cc:439] T 989bf36191dd43e490148cf0b546f529 P f5c386f2052b4c86a1c451abcc9ca25e: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7ddbaeafa194da88d61994cf63503f6" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 38089 } } peers { permanent_uuid: "bf17273192884a1e8a41b74403970a98" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 39793 } } peers { permanent_uuid: "f5c386f2052b4c86a1c451abcc9ca25e" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 43031 } } }
I20260807 18:12:29.185222 22322 tablet_replica.cc:442] T 989bf36191dd43e490148cf0b546f529 P f5c386f2052b4c86a1c451abcc9ca25e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 18:12:29.234324 22286 raft_consensus.cc:482] T 989bf36191dd43e490148cf0b546f529 P f7ddbaeafa194da88d61994cf63503f6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:12:29.234581 22286 raft_consensus.cc:504] T 989bf36191dd43e490148cf0b546f529 P f7ddbaeafa194da88d61994cf63503f6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7ddbaeafa194da88d61994cf63503f6" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 38089 } } peers { permanent_uuid: "bf17273192884a1e8a41b74403970a98" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 39793 } } peers { permanent_uuid: "f5c386f2052b4c86a1c451abcc9ca25e" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 43031 } }
I20260807 18:12:29.235316 22286 leader_election.cc:288] T 989bf36191dd43e490148cf0b546f529 P f7ddbaeafa194da88d61994cf63503f6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bf17273192884a1e8a41b74403970a98 (127.18.154.2:39793), f5c386f2052b4c86a1c451abcc9ca25e (127.18.154.3:43031)
I20260807 18:12:29.235898 21999 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "989bf36191dd43e490148cf0b546f529" candidate_uuid: "f7ddbaeafa194da88d61994cf63503f6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf17273192884a1e8a41b74403970a98" is_pre_election: true
I20260807 18:12:29.235960 22187 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "989bf36191dd43e490148cf0b546f529" candidate_uuid: "f7ddbaeafa194da88d61994cf63503f6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5c386f2052b4c86a1c451abcc9ca25e" is_pre_election: true
I20260807 18:12:29.236229 21999 raft_consensus.cc:2454] T 989bf36191dd43e490148cf0b546f529 P bf17273192884a1e8a41b74403970a98 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f7ddbaeafa194da88d61994cf63503f6 in term 0.
I20260807 18:12:29.236255 22187 raft_consensus.cc:2454] T 989bf36191dd43e490148cf0b546f529 P f5c386f2052b4c86a1c451abcc9ca25e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f7ddbaeafa194da88d61994cf63503f6 in term 0.
I20260807 18:12:29.236759 21717 leader_election.cc:302] T 989bf36191dd43e490148cf0b546f529 P f7ddbaeafa194da88d61994cf63503f6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f5c386f2052b4c86a1c451abcc9ca25e, f7ddbaeafa194da88d61994cf63503f6; no voters:
I20260807 18:12:29.237056 22286 raft_consensus.cc:2781] T 989bf36191dd43e490148cf0b546f529 P f7ddbaeafa194da88d61994cf63503f6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:12:29.237169 22286 raft_consensus.cc:482] T 989bf36191dd43e490148cf0b546f529 P f7ddbaeafa194da88d61994cf63503f6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:12:29.237254 22286 raft_consensus.cc:3037] T 989bf36191dd43e490148cf0b546f529 P f7ddbaeafa194da88d61994cf63503f6 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:12:29.239502 22286 raft_consensus.cc:504] T 989bf36191dd43e490148cf0b546f529 P f7ddbaeafa194da88d61994cf63503f6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7ddbaeafa194da88d61994cf63503f6" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 38089 } } peers { permanent_uuid: "bf17273192884a1e8a41b74403970a98" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 39793 } } peers { permanent_uuid: "f5c386f2052b4c86a1c451abcc9ca25e" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 43031 } }
I20260807 18:12:29.240065 22286 leader_election.cc:288] T 989bf36191dd43e490148cf0b546f529 P f7ddbaeafa194da88d61994cf63503f6 [CANDIDATE]: Term 1 election: Requested vote from peers bf17273192884a1e8a41b74403970a98 (127.18.154.2:39793), f5c386f2052b4c86a1c451abcc9ca25e (127.18.154.3:43031)
I20260807 18:12:29.240499 21999 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "989bf36191dd43e490148cf0b546f529" candidate_uuid: "f7ddbaeafa194da88d61994cf63503f6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf17273192884a1e8a41b74403970a98"
I20260807 18:12:29.240567 22187 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "989bf36191dd43e490148cf0b546f529" candidate_uuid: "f7ddbaeafa194da88d61994cf63503f6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5c386f2052b4c86a1c451abcc9ca25e"
I20260807 18:12:29.240758 21999 raft_consensus.cc:3037] T 989bf36191dd43e490148cf0b546f529 P bf17273192884a1e8a41b74403970a98 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:12:29.240842 22187 raft_consensus.cc:3037] T 989bf36191dd43e490148cf0b546f529 P f5c386f2052b4c86a1c451abcc9ca25e [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:12:29.242964 22187 raft_consensus.cc:2454] T 989bf36191dd43e490148cf0b546f529 P f5c386f2052b4c86a1c451abcc9ca25e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f7ddbaeafa194da88d61994cf63503f6 in term 1.
I20260807 18:12:29.242966 21999 raft_consensus.cc:2454] T 989bf36191dd43e490148cf0b546f529 P bf17273192884a1e8a41b74403970a98 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f7ddbaeafa194da88d61994cf63503f6 in term 1.
I20260807 18:12:29.243409 21717 leader_election.cc:302] T 989bf36191dd43e490148cf0b546f529 P f7ddbaeafa194da88d61994cf63503f6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f5c386f2052b4c86a1c451abcc9ca25e, f7ddbaeafa194da88d61994cf63503f6; no voters:
I20260807 18:12:29.243732 22286 raft_consensus.cc:2781] T 989bf36191dd43e490148cf0b546f529 P f7ddbaeafa194da88d61994cf63503f6 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:12:29.243988 22286 raft_consensus.cc:686] T 989bf36191dd43e490148cf0b546f529 P f7ddbaeafa194da88d61994cf63503f6 [term 1 LEADER]: Becoming Leader. State: Replica: f7ddbaeafa194da88d61994cf63503f6, State: Running, Role: LEADER
I20260807 18:12:29.244294 22286 consensus_queue.cc:237] T 989bf36191dd43e490148cf0b546f529 P f7ddbaeafa194da88d61994cf63503f6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7ddbaeafa194da88d61994cf63503f6" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 38089 } } peers { permanent_uuid: "bf17273192884a1e8a41b74403970a98" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 39793 } } peers { permanent_uuid: "f5c386f2052b4c86a1c451abcc9ca25e" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 43031 } }
I20260807 18:12:29.246148 22290 tablet_replica.cc:439] T 989bf36191dd43e490148cf0b546f529 P f7ddbaeafa194da88d61994cf63503f6: TxnStatusTablet state changed. Reason: New leader f7ddbaeafa194da88d61994cf63503f6. Latest consensus state: current_term: 1 leader_uuid: "f7ddbaeafa194da88d61994cf63503f6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7ddbaeafa194da88d61994cf63503f6" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 38089 } } peers { permanent_uuid: "bf17273192884a1e8a41b74403970a98" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 39793 } } peers { permanent_uuid: "f5c386f2052b4c86a1c451abcc9ca25e" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 43031 } } }
I20260807 18:12:29.246588 22290 tablet_replica.cc:442] T 989bf36191dd43e490148cf0b546f529 P f7ddbaeafa194da88d61994cf63503f6: This TxnStatusTablet replica's current role is: LEADER
I20260807 18:12:29.247308 22324 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260807 18:12:29.247674 22324 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260807 18:12:29.248124 21593 catalog_manager.cc:5495] T 989bf36191dd43e490148cf0b546f529 P f7ddbaeafa194da88d61994cf63503f6 reported cstate change: term changed from 0 to 1, leader changed from <none> to f7ddbaeafa194da88d61994cf63503f6 (127.18.154.1). New cstate: current_term: 1 leader_uuid: "f7ddbaeafa194da88d61994cf63503f6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7ddbaeafa194da88d61994cf63503f6" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 38089 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bf17273192884a1e8a41b74403970a98" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 39793 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5c386f2052b4c86a1c451abcc9ca25e" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 43031 } health_report { overall_health: UNKNOWN } } }
I20260807 18:12:29.281945 22327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:29.273197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55426 (local address 127.18.154.1:38089)
0807 18:12:29.274613 (+ 1416us) server_negotiation.cc:206] Beginning negotiation
0807 18:12:29.274620 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:12:29.274650 (+ 30us) server_negotiation.cc:407] Connection header received
0807 18:12:29.274755 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:12:29.274760 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:12:29.274864 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:12:29.275038 (+ 174us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:12:29.275586 (+ 548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:29.276884 (+ 1298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:29.278983 (+ 2099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:29.279742 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:29.279895 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:29.280556 (+ 661us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:12:29.281184 (+ 628us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:12:29.281312 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0807 18:12:29.281480 (+ 168us) server_negotiation.cc:299] Negotiation successful
0807 18:12:29.281610 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1271,"thread_start_us":1088,"threads_started":1}
I20260807 18:12:29.282006 22310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:29.272970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.1:38089 (local address 127.0.0.1:55426)
0807 18:12:29.273409 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0807 18:12:29.273435 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0807 18:12:29.273632 (+ 197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:12:29.275095 (+ 1463us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:12:29.275102 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:12:29.275120 (+ 18us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 18:12:29.275384 (+ 264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:12:29.275398 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:12:29.277046 (+ 1648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:12:29.277054 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:12:29.278802 (+ 1748us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:12:29.278819 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:12:29.279928 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:12:29.279936 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:12:29.280146 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:29.280272 (+ 126us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 18:12:29.281317 (+ 1045us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 18:12:29.281345 (+ 28us) client_negotiation.cc:769] Sending connection context
0807 18:12:29.281450 (+ 105us) client_negotiation.cc:240] Negotiation successful
0807 18:12:29.281658 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":220}
I20260807 18:12:29.290653 22187 raft_consensus.cc:1261] T 989bf36191dd43e490148cf0b546f529 P f5c386f2052b4c86a1c451abcc9ca25e [term 1 FOLLOWER]: Refusing update from remote peer f7ddbaeafa194da88d61994cf63503f6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:12:29.291074 21999 raft_consensus.cc:1261] T 989bf36191dd43e490148cf0b546f529 P bf17273192884a1e8a41b74403970a98 [term 1 FOLLOWER]: Refusing update from remote peer f7ddbaeafa194da88d61994cf63503f6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:12:29.291725 22290 consensus_queue.cc:1035] T 989bf36191dd43e490148cf0b546f529 P f7ddbaeafa194da88d61994cf63503f6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5c386f2052b4c86a1c451abcc9ca25e" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 43031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:12:29.292049 22286 consensus_queue.cc:1035] T 989bf36191dd43e490148cf0b546f529 P f7ddbaeafa194da88d61994cf63503f6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bf17273192884a1e8a41b74403970a98" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 39793 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:12:29.312551 22322 tablet_replica.cc:439] T 989bf36191dd43e490148cf0b546f529 P f5c386f2052b4c86a1c451abcc9ca25e: TxnStatusTablet state changed. Reason: New leader f7ddbaeafa194da88d61994cf63503f6. Latest consensus state: current_term: 1 leader_uuid: "f7ddbaeafa194da88d61994cf63503f6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7ddbaeafa194da88d61994cf63503f6" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 38089 } } peers { permanent_uuid: "bf17273192884a1e8a41b74403970a98" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 39793 } } peers { permanent_uuid: "f5c386f2052b4c86a1c451abcc9ca25e" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 43031 } } }
I20260807 18:12:29.312879 22322 tablet_replica.cc:442] T 989bf36191dd43e490148cf0b546f529 P f5c386f2052b4c86a1c451abcc9ca25e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 18:12:29.315989 22328 mvcc.cc:205] Tried to move back new op lower bound from 7315973526683484160 to 7315973526507593728. Current Snapshot: MvccSnapshot[applied={T|T < 7315973526683484160}]
I20260807 18:12:29.317107 22321 tablet_replica.cc:439] T 989bf36191dd43e490148cf0b546f529 P bf17273192884a1e8a41b74403970a98: TxnStatusTablet state changed. Reason: New leader f7ddbaeafa194da88d61994cf63503f6. Latest consensus state: current_term: 1 leader_uuid: "f7ddbaeafa194da88d61994cf63503f6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7ddbaeafa194da88d61994cf63503f6" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 38089 } } peers { permanent_uuid: "bf17273192884a1e8a41b74403970a98" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 39793 } } peers { permanent_uuid: "f5c386f2052b4c86a1c451abcc9ca25e" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 43031 } } }
I20260807 18:12:29.317435 22321 tablet_replica.cc:442] T 989bf36191dd43e490148cf0b546f529 P bf17273192884a1e8a41b74403970a98: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 18:12:29.320927 22286 tablet_replica.cc:439] T 989bf36191dd43e490148cf0b546f529 P f7ddbaeafa194da88d61994cf63503f6: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f7ddbaeafa194da88d61994cf63503f6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7ddbaeafa194da88d61994cf63503f6" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 38089 } } peers { permanent_uuid: "bf17273192884a1e8a41b74403970a98" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 39793 } } peers { permanent_uuid: "f5c386f2052b4c86a1c451abcc9ca25e" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 43031 } } }
I20260807 18:12:29.321370 22286 tablet_replica.cc:442] T 989bf36191dd43e490148cf0b546f529 P f7ddbaeafa194da88d61994cf63503f6: This TxnStatusTablet replica's current role is: LEADER
I20260807 18:12:29.327099 22330 mvcc.cc:205] Tried to move back new op lower bound from 7315973526683484160 to 7315973526507593728. Current Snapshot: MvccSnapshot[applied={T|T < 7315973526683484160}]
I20260807 18:12:29.328965 22322 tablet_replica.cc:439] T 989bf36191dd43e490148cf0b546f529 P f5c386f2052b4c86a1c451abcc9ca25e: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f7ddbaeafa194da88d61994cf63503f6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7ddbaeafa194da88d61994cf63503f6" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 38089 } } peers { permanent_uuid: "bf17273192884a1e8a41b74403970a98" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 39793 } } peers { permanent_uuid: "f5c386f2052b4c86a1c451abcc9ca25e" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 43031 } } }
I20260807 18:12:29.329228 22322 tablet_replica.cc:442] T 989bf36191dd43e490148cf0b546f529 P f5c386f2052b4c86a1c451abcc9ca25e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 18:12:29.333644 22286 tablet_replica.cc:439] T 989bf36191dd43e490148cf0b546f529 P f7ddbaeafa194da88d61994cf63503f6: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f7ddbaeafa194da88d61994cf63503f6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7ddbaeafa194da88d61994cf63503f6" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 38089 } } peers { permanent_uuid: "bf17273192884a1e8a41b74403970a98" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 39793 } } peers { permanent_uuid: "f5c386f2052b4c86a1c451abcc9ca25e" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 43031 } } }
I20260807 18:12:29.333988 22286 tablet_replica.cc:442] T 989bf36191dd43e490148cf0b546f529 P f7ddbaeafa194da88d61994cf63503f6: This TxnStatusTablet replica's current role is: LEADER
I20260807 18:12:29.338663 22321 tablet_replica.cc:439] T 989bf36191dd43e490148cf0b546f529 P bf17273192884a1e8a41b74403970a98: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f7ddbaeafa194da88d61994cf63503f6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7ddbaeafa194da88d61994cf63503f6" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 38089 } } peers { permanent_uuid: "bf17273192884a1e8a41b74403970a98" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 39793 } } peers { permanent_uuid: "f5c386f2052b4c86a1c451abcc9ca25e" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 43031 } } }
I20260807 18:12:29.338925 22321 tablet_replica.cc:442] T 989bf36191dd43e490148cf0b546f529 P bf17273192884a1e8a41b74403970a98: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 18:12:29.339465 22327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:29.333042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55436 (local address 127.18.154.1:38089)
0807 18:12:29.333221 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0807 18:12:29.333228 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:12:29.333365 (+ 137us) server_negotiation.cc:407] Connection header received
0807 18:12:29.333824 (+ 459us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:12:29.333829 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:12:29.333924 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:12:29.334013 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:12:29.334742 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:29.335872 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:29.337514 (+ 1642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:29.338025 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:29.338182 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:29.338663 (+ 481us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:12:29.338776 (+ 113us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:12:29.338897 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0807 18:12:29.339064 (+ 167us) server_negotiation.cc:299] Negotiation successful
0807 18:12:29.339157 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260807 18:12:29.358819 22327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:29.352792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.1:50887 (local address 127.18.154.1:38089)
0807 18:12:29.353055 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0807 18:12:29.353063 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:12:29.353402 (+ 339us) server_negotiation.cc:407] Connection header received
0807 18:12:29.353654 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:12:29.353659 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:12:29.353745 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:12:29.353855 (+ 110us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:12:29.354536 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:29.355434 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:29.356921 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:29.357360 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:29.357502 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:29.358005 (+ 503us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:12:29.358107 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:12:29.358204 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0807 18:12:29.358356 (+ 152us) server_negotiation.cc:299] Negotiation successful
0807 18:12:29.358421 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":24,"server-negotiator.queue_time_us":95}
I20260807 18:12:29.358881 22343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:29.352656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.1:38089 (local address 127.18.154.1:50887)
0807 18:12:29.353298 (+ 642us) negotiation.cc:107] Waiting for socket to connect
0807 18:12:29.353313 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 18:12:29.353491 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:12:29.353859 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:12:29.353863 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:12:29.353874 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 18:12:29.354384 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:12:29.354395 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:12:29.355556 (+ 1161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:12:29.355564 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:12:29.356771 (+ 1207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:12:29.356784 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:12:29.357574 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:12:29.357583 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:12:29.357717 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:29.357779 (+ 62us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 18:12:29.358203 (+ 424us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 18:12:29.358216 (+ 13us) client_negotiation.cc:769] Sending connection context
0807 18:12:29.358338 (+ 122us) client_negotiation.cc:240] Negotiation successful
0807 18:12:29.358470 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":478,"mutex_wait_us":1,"thread_start_us":307,"threads_started":1}
I20260807 18:12:29.370565 21999 raft_consensus.cc:1261] T 0605f98bc83e477ca5ed7f98effe4981 P bf17273192884a1e8a41b74403970a98 [term 1 FOLLOWER]: Refusing update from remote peer f7ddbaeafa194da88d61994cf63503f6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:12:29.370743 22187 raft_consensus.cc:1261] T 0605f98bc83e477ca5ed7f98effe4981 P f5c386f2052b4c86a1c451abcc9ca25e [term 1 FOLLOWER]: Refusing update from remote peer f7ddbaeafa194da88d61994cf63503f6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:12:29.371625 22286 consensus_queue.cc:1035] T 0605f98bc83e477ca5ed7f98effe4981 P f7ddbaeafa194da88d61994cf63503f6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5c386f2052b4c86a1c451abcc9ca25e" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 43031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:12:29.371912 22344 consensus_queue.cc:1035] T 0605f98bc83e477ca5ed7f98effe4981 P f7ddbaeafa194da88d61994cf63503f6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bf17273192884a1e8a41b74403970a98" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 39793 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:12:29.464781 21593 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58836:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:12:29.465088 21593 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58836:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:12:29.468974 21593 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6fff555ae1b94e1494e4fb470dad5674: Sending DeleteTablet for 3 replicas of tablet 0605f98bc83e477ca5ed7f98effe4981
I20260807 18:12:29.470125 21783 tablet_service.cc:1514] Processing DeleteTablet for tablet 0605f98bc83e477ca5ed7f98effe4981 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:12:29 UTC) from {username='kudu', principal='kudu/127.18.154.62@KRBTEST.COM'} at 127.0.0.1:55412
I20260807 18:12:29.470291 22163 tablet_service.cc:1514] Processing DeleteTablet for tablet 0605f98bc83e477ca5ed7f98effe4981 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:12:29 UTC) from {username='kudu', principal='kudu/127.18.154.62@KRBTEST.COM'} at 127.0.0.1:45078
I20260807 18:12:29.470564 21970 tablet_service.cc:1514] Processing DeleteTablet for tablet 0605f98bc83e477ca5ed7f98effe4981 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:12:29 UTC) from {username='kudu', principal='kudu/127.18.154.62@KRBTEST.COM'} at 127.0.0.1:36944
I20260807 18:12:29.472844 21593 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58836:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 18:12:29.473829 21593 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:12:29.482268 21967 tablet_service.cc:1467] Processing CreateTablet for tablet 58acef6c1862469fbb3450dcd7f3e992 (DEFAULT_TABLE table=test-table [id=23b6cd56aa434d2490cb067cfafd8ec7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:12:29.482455 22160 tablet_service.cc:1467] Processing CreateTablet for tablet 58acef6c1862469fbb3450dcd7f3e992 (DEFAULT_TABLE table=test-table [id=23b6cd56aa434d2490cb067cfafd8ec7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:12:29.482627 22350 tablet_replica.cc:330] T 0605f98bc83e477ca5ed7f98effe4981 P f7ddbaeafa194da88d61994cf63503f6: stopping tablet replica
I20260807 18:12:29.482935 21967 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 58acef6c1862469fbb3450dcd7f3e992. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:12:29.482998 22160 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 58acef6c1862469fbb3450dcd7f3e992. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:12:29.483213 22350 raft_consensus.cc:2229] T 0605f98bc83e477ca5ed7f98effe4981 P f7ddbaeafa194da88d61994cf63503f6 [term 1 LEADER]: Raft consensus shutting down.
I20260807 18:12:29.483512 22351 tablet_replica.cc:330] T 0605f98bc83e477ca5ed7f98effe4981 P bf17273192884a1e8a41b74403970a98: stopping tablet replica
I20260807 18:12:29.483937 22350 raft_consensus.cc:2258] T 0605f98bc83e477ca5ed7f98effe4981 P f7ddbaeafa194da88d61994cf63503f6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:12:29.483954 22351 raft_consensus.cc:2229] T 0605f98bc83e477ca5ed7f98effe4981 P bf17273192884a1e8a41b74403970a98 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:12:29.484411 22351 raft_consensus.cc:2258] T 0605f98bc83e477ca5ed7f98effe4981 P bf17273192884a1e8a41b74403970a98 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:12:29.485502 22349 tablet_replica.cc:330] T 0605f98bc83e477ca5ed7f98effe4981 P f5c386f2052b4c86a1c451abcc9ca25e: stopping tablet replica
I20260807 18:12:29.485929 22349 raft_consensus.cc:2229] T 0605f98bc83e477ca5ed7f98effe4981 P f5c386f2052b4c86a1c451abcc9ca25e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:12:29.486347 22349 raft_consensus.cc:2258] T 0605f98bc83e477ca5ed7f98effe4981 P f5c386f2052b4c86a1c451abcc9ca25e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:12:29.486563 21784 tablet_service.cc:1467] Processing CreateTablet for tablet 58acef6c1862469fbb3450dcd7f3e992 (DEFAULT_TABLE table=test-table [id=23b6cd56aa434d2490cb067cfafd8ec7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:12:29.487005 21784 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 58acef6c1862469fbb3450dcd7f3e992. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:12:29.487100 22351 ts_tablet_manager.cc:1905] T 0605f98bc83e477ca5ed7f98effe4981 P bf17273192884a1e8a41b74403970a98: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:12:29.488065 22350 ts_tablet_manager.cc:1905] T 0605f98bc83e477ca5ed7f98effe4981 P f7ddbaeafa194da88d61994cf63503f6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:12:29.490244 22313 tablet_bootstrap.cc:492] T 58acef6c1862469fbb3450dcd7f3e992 P bf17273192884a1e8a41b74403970a98: Bootstrap starting.
I20260807 18:12:29.491991 22313 tablet_bootstrap.cc:654] T 58acef6c1862469fbb3450dcd7f3e992 P bf17273192884a1e8a41b74403970a98: Neither blocks nor log segments found. Creating new log.
I20260807 18:12:29.495051 22351 ts_tablet_manager.cc:1918] T 0605f98bc83e477ca5ed7f98effe4981 P bf17273192884a1e8a41b74403970a98: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 18:12:29.495312 22351 log.cc:1196] T 0605f98bc83e477ca5ed7f98effe4981 P bf17273192884a1e8a41b74403970a98: Deleting WAL directory at /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126328808105-19048-0/minicluster-data/ts-1/wal/wals/0605f98bc83e477ca5ed7f98effe4981
I20260807 18:12:29.495791 22350 ts_tablet_manager.cc:1918] T 0605f98bc83e477ca5ed7f98effe4981 P f7ddbaeafa194da88d61994cf63503f6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 18:12:29.496098 22351 ts_tablet_manager.cc:1939] T 0605f98bc83e477ca5ed7f98effe4981 P bf17273192884a1e8a41b74403970a98: Deleting consensus metadata
I20260807 18:12:29.496089 22350 log.cc:1196] T 0605f98bc83e477ca5ed7f98effe4981 P f7ddbaeafa194da88d61994cf63503f6: Deleting WAL directory at /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126328808105-19048-0/minicluster-data/ts-0/wal/wals/0605f98bc83e477ca5ed7f98effe4981
I20260807 18:12:29.496644 22350 ts_tablet_manager.cc:1939] T 0605f98bc83e477ca5ed7f98effe4981 P f7ddbaeafa194da88d61994cf63503f6: Deleting consensus metadata
I20260807 18:12:29.498492 21572 catalog_manager.cc:4841] TS bf17273192884a1e8a41b74403970a98 (127.18.154.2:39793): tablet 0605f98bc83e477ca5ed7f98effe4981 (table test-table [id=cae4ad4aadb64a7780008ac6b6fbd4ed]) successfully deleted
I20260807 18:12:29.498824 21575 catalog_manager.cc:4841] TS f7ddbaeafa194da88d61994cf63503f6 (127.18.154.1:38089): tablet 0605f98bc83e477ca5ed7f98effe4981 (table test-table [id=cae4ad4aadb64a7780008ac6b6fbd4ed]) successfully deleted
I20260807 18:12:29.502338 22315 tablet_bootstrap.cc:492] T 58acef6c1862469fbb3450dcd7f3e992 P f5c386f2052b4c86a1c451abcc9ca25e: Bootstrap starting.
I20260807 18:12:29.504874 22315 tablet_bootstrap.cc:654] T 58acef6c1862469fbb3450dcd7f3e992 P f5c386f2052b4c86a1c451abcc9ca25e: Neither blocks nor log segments found. Creating new log.
I20260807 18:12:29.505235 22349 ts_tablet_manager.cc:1905] T 0605f98bc83e477ca5ed7f98effe4981 P f5c386f2052b4c86a1c451abcc9ca25e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:12:29.513278 22349 ts_tablet_manager.cc:1918] T 0605f98bc83e477ca5ed7f98effe4981 P f5c386f2052b4c86a1c451abcc9ca25e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 18:12:29.513492 22349 log.cc:1196] T 0605f98bc83e477ca5ed7f98effe4981 P f5c386f2052b4c86a1c451abcc9ca25e: Deleting WAL directory at /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126328808105-19048-0/minicluster-data/ts-2/wal/wals/0605f98bc83e477ca5ed7f98effe4981
I20260807 18:12:29.514216 22312 tablet_bootstrap.cc:492] T 58acef6c1862469fbb3450dcd7f3e992 P f7ddbaeafa194da88d61994cf63503f6: Bootstrap starting.
I20260807 18:12:29.514828 22349 ts_tablet_manager.cc:1939] T 0605f98bc83e477ca5ed7f98effe4981 P f5c386f2052b4c86a1c451abcc9ca25e: Deleting consensus metadata
I20260807 18:12:29.516714 22312 tablet_bootstrap.cc:654] T 58acef6c1862469fbb3450dcd7f3e992 P f7ddbaeafa194da88d61994cf63503f6: Neither blocks nor log segments found. Creating new log.
I20260807 18:12:29.517518 21571 catalog_manager.cc:4841] TS f5c386f2052b4c86a1c451abcc9ca25e (127.18.154.3:43031): tablet 0605f98bc83e477ca5ed7f98effe4981 (table test-table [id=cae4ad4aadb64a7780008ac6b6fbd4ed]) successfully deleted
I20260807 18:12:29.519052 22315 tablet_bootstrap.cc:492] T 58acef6c1862469fbb3450dcd7f3e992 P f5c386f2052b4c86a1c451abcc9ca25e: No bootstrap required, opened a new log
I20260807 18:12:29.519239 22313 tablet_bootstrap.cc:492] T 58acef6c1862469fbb3450dcd7f3e992 P bf17273192884a1e8a41b74403970a98: No bootstrap required, opened a new log
I20260807 18:12:29.519323 22315 ts_tablet_manager.cc:1397] T 58acef6c1862469fbb3450dcd7f3e992 P f5c386f2052b4c86a1c451abcc9ca25e: Time spent bootstrapping tablet: real 0.017s user 0.004s sys 0.004s
I20260807 18:12:29.519460 22313 ts_tablet_manager.cc:1397] T 58acef6c1862469fbb3450dcd7f3e992 P bf17273192884a1e8a41b74403970a98: Time spent bootstrapping tablet: real 0.029s user 0.004s sys 0.020s
I20260807 18:12:29.520128 22313 raft_consensus.cc:348] T 58acef6c1862469fbb3450dcd7f3e992 P bf17273192884a1e8a41b74403970a98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7ddbaeafa194da88d61994cf63503f6" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 38089 } } peers { permanent_uuid: "bf17273192884a1e8a41b74403970a98" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 39793 } } peers { permanent_uuid: "f5c386f2052b4c86a1c451abcc9ca25e" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 43031 } }
I20260807 18:12:29.520370 22313 raft_consensus.cc:374] T 58acef6c1862469fbb3450dcd7f3e992 P bf17273192884a1e8a41b74403970a98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:12:29.520282 22315 raft_consensus.cc:348] T 58acef6c1862469fbb3450dcd7f3e992 P f5c386f2052b4c86a1c451abcc9ca25e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7ddbaeafa194da88d61994cf63503f6" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 38089 } } peers { permanent_uuid: "bf17273192884a1e8a41b74403970a98" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 39793 } } peers { permanent_uuid: "f5c386f2052b4c86a1c451abcc9ca25e" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 43031 } }
I20260807 18:12:29.520455 22313 raft_consensus.cc:729] T 58acef6c1862469fbb3450dcd7f3e992 P bf17273192884a1e8a41b74403970a98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf17273192884a1e8a41b74403970a98, State: Initialized, Role: FOLLOWER
I20260807 18:12:29.520530 22315 raft_consensus.cc:374] T 58acef6c1862469fbb3450dcd7f3e992 P f5c386f2052b4c86a1c451abcc9ca25e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:12:29.520610 22315 raft_consensus.cc:729] T 58acef6c1862469fbb3450dcd7f3e992 P f5c386f2052b4c86a1c451abcc9ca25e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5c386f2052b4c86a1c451abcc9ca25e, State: Initialized, Role: FOLLOWER
I20260807 18:12:29.520833 22313 consensus_queue.cc:260] T 58acef6c1862469fbb3450dcd7f3e992 P bf17273192884a1e8a41b74403970a98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7ddbaeafa194da88d61994cf63503f6" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 38089 } } peers { permanent_uuid: "bf17273192884a1e8a41b74403970a98" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 39793 } } peers { permanent_uuid: "f5c386f2052b4c86a1c451abcc9ca25e" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 43031 } }
I20260807 18:12:29.520848 22315 consensus_queue.cc:260] T 58acef6c1862469fbb3450dcd7f3e992 P f5c386f2052b4c86a1c451abcc9ca25e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7ddbaeafa194da88d61994cf63503f6" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 38089 } } peers { permanent_uuid: "bf17273192884a1e8a41b74403970a98" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 39793 } } peers { permanent_uuid: "f5c386f2052b4c86a1c451abcc9ca25e" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 43031 } }
I20260807 18:12:29.521488 22315 ts_tablet_manager.cc:1428] T 58acef6c1862469fbb3450dcd7f3e992 P f5c386f2052b4c86a1c451abcc9ca25e: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260807 18:12:29.521507 22313 ts_tablet_manager.cc:1428] T 58acef6c1862469fbb3450dcd7f3e992 P bf17273192884a1e8a41b74403970a98: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260807 18:12:29.525622 22312 tablet_bootstrap.cc:492] T 58acef6c1862469fbb3450dcd7f3e992 P f7ddbaeafa194da88d61994cf63503f6: No bootstrap required, opened a new log
I20260807 18:12:29.525807 22312 ts_tablet_manager.cc:1397] T 58acef6c1862469fbb3450dcd7f3e992 P f7ddbaeafa194da88d61994cf63503f6: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.006s
I20260807 18:12:29.526600 22312 raft_consensus.cc:348] T 58acef6c1862469fbb3450dcd7f3e992 P f7ddbaeafa194da88d61994cf63503f6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7ddbaeafa194da88d61994cf63503f6" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 38089 } } peers { permanent_uuid: "bf17273192884a1e8a41b74403970a98" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 39793 } } peers { permanent_uuid: "f5c386f2052b4c86a1c451abcc9ca25e" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 43031 } }
I20260807 18:12:29.526901 22312 raft_consensus.cc:374] T 58acef6c1862469fbb3450dcd7f3e992 P f7ddbaeafa194da88d61994cf63503f6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:12:29.526984 22312 raft_consensus.cc:729] T 58acef6c1862469fbb3450dcd7f3e992 P f7ddbaeafa194da88d61994cf63503f6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7ddbaeafa194da88d61994cf63503f6, State: Initialized, Role: FOLLOWER
I20260807 18:12:29.527249 22312 consensus_queue.cc:260] T 58acef6c1862469fbb3450dcd7f3e992 P f7ddbaeafa194da88d61994cf63503f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7ddbaeafa194da88d61994cf63503f6" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 38089 } } peers { permanent_uuid: "bf17273192884a1e8a41b74403970a98" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 39793 } } peers { permanent_uuid: "f5c386f2052b4c86a1c451abcc9ca25e" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 43031 } }
I20260807 18:12:29.527833 22312 ts_tablet_manager.cc:1428] T 58acef6c1862469fbb3450dcd7f3e992 P f7ddbaeafa194da88d61994cf63503f6: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260807 18:12:29.536281 22322 raft_consensus.cc:482] T 58acef6c1862469fbb3450dcd7f3e992 P f5c386f2052b4c86a1c451abcc9ca25e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:12:29.536509 22322 raft_consensus.cc:504] T 58acef6c1862469fbb3450dcd7f3e992 P f5c386f2052b4c86a1c451abcc9ca25e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7ddbaeafa194da88d61994cf63503f6" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 38089 } } peers { permanent_uuid: "bf17273192884a1e8a41b74403970a98" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 39793 } } peers { permanent_uuid: "f5c386f2052b4c86a1c451abcc9ca25e" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 43031 } }
I20260807 18:12:29.538244 22322 leader_election.cc:288] T 58acef6c1862469fbb3450dcd7f3e992 P f5c386f2052b4c86a1c451abcc9ca25e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f7ddbaeafa194da88d61994cf63503f6 (127.18.154.1:38089), bf17273192884a1e8a41b74403970a98 (127.18.154.2:39793)
I20260807 18:12:29.559589 22356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:29.538450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.1:38089 (local address 127.18.154.3:54711)
0807 18:12:29.552009 (+ 13559us) negotiation.cc:107] Waiting for socket to connect
0807 18:12:29.552033 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 18:12:29.552393 (+ 360us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:12:29.552964 (+ 571us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:12:29.552967 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:12:29.552989 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:12:29.553322 (+ 333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:12:29.553332 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:12:29.554924 (+ 1592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:12:29.554936 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:12:29.557251 (+ 2315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:12:29.557269 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:12:29.558782 (+ 1513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:12:29.558788 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:12:29.558957 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:29.558981 (+ 24us) client_negotiation.cc:769] Sending connection context
0807 18:12:29.559096 (+ 115us) client_negotiation.cc:240] Negotiation successful
0807 18:12:29.559233 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13417,"thread_start_us":15194,"threads_started":1}
I20260807 18:12:29.559551 22327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:29.538703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.3:54711 (local address 127.18.154.1:38089)
0807 18:12:29.538960 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0807 18:12:29.538971 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 18:12:29.552132 (+ 13161us) server_negotiation.cc:407] Connection header received
0807 18:12:29.552594 (+ 462us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:12:29.552601 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:12:29.552694 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:12:29.552775 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:12:29.553503 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:29.554719 (+ 1216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:29.557445 (+ 2726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:29.558605 (+ 1160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:29.558790 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:29.558868 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0807 18:12:29.559125 (+ 257us) server_negotiation.cc:299] Negotiation successful
0807 18:12:29.559266 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260807 18:12:29.560649 22355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:29.538588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.2:39793 (local address 127.18.154.3:55021)
0807 18:12:29.553976 (+ 15388us) negotiation.cc:107] Waiting for socket to connect
0807 18:12:29.553995 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 18:12:29.554114 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:12:29.554485 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:12:29.554490 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:12:29.554515 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:12:29.555000 (+ 485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:12:29.555014 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:12:29.556261 (+ 1247us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:12:29.556269 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:12:29.557921 (+ 1652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:12:29.557936 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:12:29.559783 (+ 1847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:12:29.559789 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:12:29.559990 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:29.560026 (+ 36us) client_negotiation.cc:769] Sending connection context
0807 18:12:29.560138 (+ 112us) client_negotiation.cc:240] Negotiation successful
0807 18:12:29.560274 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15270,"mutex_wait_us":22,"thread_start_us":13137,"threads_started":1}
I20260807 18:12:29.560984 21814 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "58acef6c1862469fbb3450dcd7f3e992" candidate_uuid: "f5c386f2052b4c86a1c451abcc9ca25e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7ddbaeafa194da88d61994cf63503f6" is_pre_election: true
I20260807 18:12:29.561494 22357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:29.539222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.3:55021 (local address 127.18.154.2:39793)
0807 18:12:29.551531 (+ 12309us) server_negotiation.cc:206] Beginning negotiation
0807 18:12:29.551540 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:12:29.554113 (+ 2573us) server_negotiation.cc:407] Connection header received
0807 18:12:29.554220 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:12:29.554226 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:12:29.554340 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:12:29.554453 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:12:29.555189 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:29.556122 (+ 933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:29.558104 (+ 1982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:29.559395 (+ 1291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:29.559557 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:29.559647 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0807 18:12:29.560923 (+ 1276us) server_negotiation.cc:299] Negotiation successful
0807 18:12:29.561077 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12139,"thread_start_us":12930,"threads_started":1}
I20260807 18:12:29.561465 21814 raft_consensus.cc:2454] T 58acef6c1862469fbb3450dcd7f3e992 P f7ddbaeafa194da88d61994cf63503f6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5c386f2052b4c86a1c451abcc9ca25e in term 0.
I20260807 18:12:29.562255 22097 leader_election.cc:302] T 58acef6c1862469fbb3450dcd7f3e992 P f5c386f2052b4c86a1c451abcc9ca25e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f5c386f2052b4c86a1c451abcc9ca25e, f7ddbaeafa194da88d61994cf63503f6; no voters:
I20260807 18:12:29.562372 21999 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "58acef6c1862469fbb3450dcd7f3e992" candidate_uuid: "f5c386f2052b4c86a1c451abcc9ca25e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf17273192884a1e8a41b74403970a98" is_pre_election: true
I20260807 18:12:29.562623 21999 raft_consensus.cc:2454] T 58acef6c1862469fbb3450dcd7f3e992 P bf17273192884a1e8a41b74403970a98 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5c386f2052b4c86a1c451abcc9ca25e in term 0.
I20260807 18:12:29.562739 22322 raft_consensus.cc:2781] T 58acef6c1862469fbb3450dcd7f3e992 P f5c386f2052b4c86a1c451abcc9ca25e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:12:29.562857 22322 raft_consensus.cc:482] T 58acef6c1862469fbb3450dcd7f3e992 P f5c386f2052b4c86a1c451abcc9ca25e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:12:29.562969 22322 raft_consensus.cc:3037] T 58acef6c1862469fbb3450dcd7f3e992 P f5c386f2052b4c86a1c451abcc9ca25e [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:12:29.565620 22322 raft_consensus.cc:504] T 58acef6c1862469fbb3450dcd7f3e992 P f5c386f2052b4c86a1c451abcc9ca25e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7ddbaeafa194da88d61994cf63503f6" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 38089 } } peers { permanent_uuid: "bf17273192884a1e8a41b74403970a98" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 39793 } } peers { permanent_uuid: "f5c386f2052b4c86a1c451abcc9ca25e" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 43031 } }
I20260807 18:12:29.566309 22322 leader_election.cc:288] T 58acef6c1862469fbb3450dcd7f3e992 P f5c386f2052b4c86a1c451abcc9ca25e [CANDIDATE]: Term 1 election: Requested vote from peers f7ddbaeafa194da88d61994cf63503f6 (127.18.154.1:38089), bf17273192884a1e8a41b74403970a98 (127.18.154.2:39793)
I20260807 18:12:29.566797 21814 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "58acef6c1862469fbb3450dcd7f3e992" candidate_uuid: "f5c386f2052b4c86a1c451abcc9ca25e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7ddbaeafa194da88d61994cf63503f6"
I20260807 18:12:29.566993 21814 raft_consensus.cc:3037] T 58acef6c1862469fbb3450dcd7f3e992 P f7ddbaeafa194da88d61994cf63503f6 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:12:29.567005 21999 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "58acef6c1862469fbb3450dcd7f3e992" candidate_uuid: "f5c386f2052b4c86a1c451abcc9ca25e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf17273192884a1e8a41b74403970a98"
I20260807 18:12:29.567251 21999 raft_consensus.cc:3037] T 58acef6c1862469fbb3450dcd7f3e992 P bf17273192884a1e8a41b74403970a98 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:12:29.569279 21999 raft_consensus.cc:2454] T 58acef6c1862469fbb3450dcd7f3e992 P bf17273192884a1e8a41b74403970a98 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5c386f2052b4c86a1c451abcc9ca25e in term 1.
I20260807 18:12:29.569773 22095 leader_election.cc:302] T 58acef6c1862469fbb3450dcd7f3e992 P f5c386f2052b4c86a1c451abcc9ca25e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bf17273192884a1e8a41b74403970a98, f5c386f2052b4c86a1c451abcc9ca25e; no voters:
I20260807 18:12:29.570133 22322 raft_consensus.cc:2781] T 58acef6c1862469fbb3450dcd7f3e992 P f5c386f2052b4c86a1c451abcc9ca25e [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:12:29.570094 21814 raft_consensus.cc:2454] T 58acef6c1862469fbb3450dcd7f3e992 P f7ddbaeafa194da88d61994cf63503f6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5c386f2052b4c86a1c451abcc9ca25e in term 1.
I20260807 18:12:29.570691 22322 raft_consensus.cc:686] T 58acef6c1862469fbb3450dcd7f3e992 P f5c386f2052b4c86a1c451abcc9ca25e [term 1 LEADER]: Becoming Leader. State: Replica: f5c386f2052b4c86a1c451abcc9ca25e, State: Running, Role: LEADER
I20260807 18:12:29.571111 22322 consensus_queue.cc:237] T 58acef6c1862469fbb3450dcd7f3e992 P f5c386f2052b4c86a1c451abcc9ca25e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7ddbaeafa194da88d61994cf63503f6" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 38089 } } peers { permanent_uuid: "bf17273192884a1e8a41b74403970a98" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 39793 } } peers { permanent_uuid: "f5c386f2052b4c86a1c451abcc9ca25e" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 43031 } }
I20260807 18:12:29.574913 21592 catalog_manager.cc:5495] T 58acef6c1862469fbb3450dcd7f3e992 P f5c386f2052b4c86a1c451abcc9ca25e reported cstate change: term changed from 0 to 1, leader changed from <none> to f5c386f2052b4c86a1c451abcc9ca25e (127.18.154.3). New cstate: current_term: 1 leader_uuid: "f5c386f2052b4c86a1c451abcc9ca25e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7ddbaeafa194da88d61994cf63503f6" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 38089 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bf17273192884a1e8a41b74403970a98" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 39793 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5c386f2052b4c86a1c451abcc9ca25e" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 43031 } health_report { overall_health: HEALTHY } } }
I20260807 18:12:29.615291 22362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:29.608646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45082 (local address 127.18.154.3:43031)
0807 18:12:29.609280 (+ 634us) server_negotiation.cc:206] Beginning negotiation
0807 18:12:29.609285 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:12:29.609303 (+ 18us) server_negotiation.cc:407] Connection header received
0807 18:12:29.609428 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:12:29.609433 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:12:29.609561 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:12:29.609758 (+ 197us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:12:29.610217 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:29.610995 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:29.612358 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:29.612791 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:29.612928 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:29.613410 (+ 482us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:12:29.614378 (+ 968us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:12:29.614534 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0807 18:12:29.614644 (+ 110us) server_negotiation.cc:299] Negotiation successful
0807 18:12:29.614729 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":484,"thread_start_us":318,"threads_started":1}
I20260807 18:12:29.633203 22327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:29.627024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.3:53777 (local address 127.18.154.1:38089)
0807 18:12:29.627311 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0807 18:12:29.627320 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:12:29.627926 (+ 606us) server_negotiation.cc:407] Connection header received
0807 18:12:29.628163 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:12:29.628167 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:12:29.628240 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:12:29.628345 (+ 105us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:12:29.628923 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:29.629749 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:29.631150 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:29.631677 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:29.631851 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:29.632353 (+ 502us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:12:29.632488 (+ 135us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:12:29.632598 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0807 18:12:29.632760 (+ 162us) server_negotiation.cc:299] Negotiation successful
0807 18:12:29.632856 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260807 18:12:29.633276 22366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:29.626851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.1:38089 (local address 127.18.154.3:53777)
0807 18:12:29.627819 (+ 968us) negotiation.cc:107] Waiting for socket to connect
0807 18:12:29.627835 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 18:12:29.628006 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:12:29.628363 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:12:29.628368 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:12:29.628380 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 18:12:29.628739 (+ 359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:12:29.628760 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:12:29.629888 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:12:29.629894 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:12:29.631000 (+ 1106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:12:29.631012 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:12:29.631919 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:12:29.631926 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:12:29.632081 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:29.632145 (+ 64us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 18:12:29.632600 (+ 455us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 18:12:29.632615 (+ 15us) client_negotiation.cc:769] Sending connection context
0807 18:12:29.632723 (+ 108us) client_negotiation.cc:240] Negotiation successful
0807 18:12:29.632876 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":808,"thread_start_us":696,"threads_started":1}
I20260807 18:12:29.643718 21999 raft_consensus.cc:1261] T 58acef6c1862469fbb3450dcd7f3e992 P bf17273192884a1e8a41b74403970a98 [term 1 FOLLOWER]: Refusing update from remote peer f5c386f2052b4c86a1c451abcc9ca25e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:12:29.644155 21814 raft_consensus.cc:1261] T 58acef6c1862469fbb3450dcd7f3e992 P f7ddbaeafa194da88d61994cf63503f6 [term 1 FOLLOWER]: Refusing update from remote peer f5c386f2052b4c86a1c451abcc9ca25e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:12:29.645053 22322 consensus_queue.cc:1035] T 58acef6c1862469fbb3450dcd7f3e992 P f5c386f2052b4c86a1c451abcc9ca25e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f7ddbaeafa194da88d61994cf63503f6" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 38089 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:12:29.645443 22360 consensus_queue.cc:1035] T 58acef6c1862469fbb3450dcd7f3e992 P f5c386f2052b4c86a1c451abcc9ca25e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bf17273192884a1e8a41b74403970a98" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 39793 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:12:29.657701 22329 mvcc.cc:205] Tried to move back new op lower bound from 7315973528133636096 to 7315973527847264256. Current Snapshot: MvccSnapshot[applied={T|T < 7315973528133636096}]
I20260807 18:12:29.691407 22362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:29.684943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.1:46883 (local address 127.18.154.3:43031)
0807 18:12:29.685129 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0807 18:12:29.685135 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:12:29.685152 (+ 17us) server_negotiation.cc:407] Connection header received
0807 18:12:29.685231 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:12:29.685240 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:12:29.685352 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:12:29.685451 (+ 99us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:12:29.685979 (+ 528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:29.687198 (+ 1219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:29.689275 (+ 2077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:29.689978 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:29.690114 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:29.690583 (+ 469us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:12:29.690728 (+ 145us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:12:29.690826 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0807 18:12:29.691008 (+ 182us) server_negotiation.cc:299] Negotiation successful
0807 18:12:29.691088 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260807 18:12:29.691457 22343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:29.684507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.3:43031 (local address 127.18.154.1:46883)
0807 18:12:29.684704 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0807 18:12:29.684726 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 18:12:29.684877 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:12:29.685512 (+ 635us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:12:29.685518 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:12:29.685568 (+ 50us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 18:12:29.685794 (+ 226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:12:29.685806 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:12:29.687368 (+ 1562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:12:29.687373 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:12:29.689104 (+ 1731us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:12:29.689119 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:12:29.690153 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:12:29.690158 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:12:29.690374 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:29.690424 (+ 50us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 18:12:29.690850 (+ 426us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 18:12:29.690862 (+ 12us) client_negotiation.cc:769] Sending connection context
0807 18:12:29.690984 (+ 122us) client_negotiation.cc:240] Negotiation successful
0807 18:12:29.691126 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
W20260807 18:12:29.692654 22329 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260807 18:12:31.314291 21592 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58836:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:12:31.314601 21592 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58836:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:12:31.319031 21592 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6fff555ae1b94e1494e4fb470dad5674: Sending DeleteTablet for 3 replicas of tablet 58acef6c1862469fbb3450dcd7f3e992
I20260807 18:12:31.319854 21784 tablet_service.cc:1514] Processing DeleteTablet for tablet 58acef6c1862469fbb3450dcd7f3e992 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:12:31 UTC) from {username='kudu', principal='kudu/127.18.154.62@KRBTEST.COM'} at 127.0.0.1:55412
I20260807 18:12:31.319948 21967 tablet_service.cc:1514] Processing DeleteTablet for tablet 58acef6c1862469fbb3450dcd7f3e992 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:12:31 UTC) from {username='kudu', principal='kudu/127.18.154.62@KRBTEST.COM'} at 127.0.0.1:36944
I20260807 18:12:31.320235 22160 tablet_service.cc:1514] Processing DeleteTablet for tablet 58acef6c1862469fbb3450dcd7f3e992 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:12:31 UTC) from {username='kudu', principal='kudu/127.18.154.62@KRBTEST.COM'} at 127.0.0.1:45078
I20260807 18:12:31.333469 22404 tablet_replica.cc:330] T 58acef6c1862469fbb3450dcd7f3e992 P f5c386f2052b4c86a1c451abcc9ca25e: stopping tablet replica
I20260807 18:12:31.334404 22404 raft_consensus.cc:2229] T 58acef6c1862469fbb3450dcd7f3e992 P f5c386f2052b4c86a1c451abcc9ca25e [term 1 LEADER]: Raft consensus shutting down.
I20260807 18:12:31.334978 22404 raft_consensus.cc:2258] T 58acef6c1862469fbb3450dcd7f3e992 P f5c386f2052b4c86a1c451abcc9ca25e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:12:31.336727 22402 tablet_replica.cc:330] T 58acef6c1862469fbb3450dcd7f3e992 P bf17273192884a1e8a41b74403970a98: stopping tablet replica
I20260807 18:12:31.336992 22404 ts_tablet_manager.cc:1905] T 58acef6c1862469fbb3450dcd7f3e992 P f5c386f2052b4c86a1c451abcc9ca25e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:12:31.337160 22403 tablet_replica.cc:330] T 58acef6c1862469fbb3450dcd7f3e992 P f7ddbaeafa194da88d61994cf63503f6: stopping tablet replica
I20260807 18:12:31.337468 22403 raft_consensus.cc:2229] T 58acef6c1862469fbb3450dcd7f3e992 P f7ddbaeafa194da88d61994cf63503f6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:12:31.337844 22402 raft_consensus.cc:2229] T 58acef6c1862469fbb3450dcd7f3e992 P bf17273192884a1e8a41b74403970a98 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:12:31.338178 22403 raft_consensus.cc:2258] T 58acef6c1862469fbb3450dcd7f3e992 P f7ddbaeafa194da88d61994cf63503f6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:12:31.338505 22402 raft_consensus.cc:2258] T 58acef6c1862469fbb3450dcd7f3e992 P bf17273192884a1e8a41b74403970a98 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:12:31.339862 22402 ts_tablet_manager.cc:1905] T 58acef6c1862469fbb3450dcd7f3e992 P bf17273192884a1e8a41b74403970a98: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:12:31.342880 22404 ts_tablet_manager.cc:1918] T 58acef6c1862469fbb3450dcd7f3e992 P f5c386f2052b4c86a1c451abcc9ca25e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 18:12:31.343034 22404 log.cc:1196] T 58acef6c1862469fbb3450dcd7f3e992 P f5c386f2052b4c86a1c451abcc9ca25e: Deleting WAL directory at /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126328808105-19048-0/minicluster-data/ts-2/wal/wals/58acef6c1862469fbb3450dcd7f3e992
I20260807 18:12:31.343603 22404 ts_tablet_manager.cc:1939] T 58acef6c1862469fbb3450dcd7f3e992 P f5c386f2052b4c86a1c451abcc9ca25e: Deleting consensus metadata
I20260807 18:12:31.344084 22403 ts_tablet_manager.cc:1905] T 58acef6c1862469fbb3450dcd7f3e992 P f7ddbaeafa194da88d61994cf63503f6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:12:31.344996 21571 catalog_manager.cc:4841] TS f5c386f2052b4c86a1c451abcc9ca25e (127.18.154.3:43031): tablet 58acef6c1862469fbb3450dcd7f3e992 (table test-table [id=23b6cd56aa434d2490cb067cfafd8ec7]) successfully deleted
I20260807 18:12:31.347762 22402 ts_tablet_manager.cc:1918] T 58acef6c1862469fbb3450dcd7f3e992 P bf17273192884a1e8a41b74403970a98: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 18:12:31.347978 22402 log.cc:1196] T 58acef6c1862469fbb3450dcd7f3e992 P bf17273192884a1e8a41b74403970a98: Deleting WAL directory at /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126328808105-19048-0/minicluster-data/ts-1/wal/wals/58acef6c1862469fbb3450dcd7f3e992
I20260807 18:12:31.348569 22402 ts_tablet_manager.cc:1939] T 58acef6c1862469fbb3450dcd7f3e992 P bf17273192884a1e8a41b74403970a98: Deleting consensus metadata
I20260807 18:12:31.349707 21572 catalog_manager.cc:4841] TS bf17273192884a1e8a41b74403970a98 (127.18.154.2:39793): tablet 58acef6c1862469fbb3450dcd7f3e992 (table test-table [id=23b6cd56aa434d2490cb067cfafd8ec7]) successfully deleted
I20260807 18:12:31.351899 22403 ts_tablet_manager.cc:1918] T 58acef6c1862469fbb3450dcd7f3e992 P f7ddbaeafa194da88d61994cf63503f6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 18:12:31.352054 22403 log.cc:1196] T 58acef6c1862469fbb3450dcd7f3e992 P f7ddbaeafa194da88d61994cf63503f6: Deleting WAL directory at /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126328808105-19048-0/minicluster-data/ts-0/wal/wals/58acef6c1862469fbb3450dcd7f3e992
I20260807 18:12:31.352633 22403 ts_tablet_manager.cc:1939] T 58acef6c1862469fbb3450dcd7f3e992 P f7ddbaeafa194da88d61994cf63503f6: Deleting consensus metadata
I20260807 18:12:31.354168 21575 catalog_manager.cc:4841] TS f7ddbaeafa194da88d61994cf63503f6 (127.18.154.1:38089): tablet 58acef6c1862469fbb3450dcd7f3e992 (table test-table [id=23b6cd56aa434d2490cb067cfafd8ec7]) successfully deleted
Aug 07 18:12:31 dist-test-slave-gkw7 krb5kdc[21531](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126351, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 18:12:31 dist-test-slave-gkw7 krb5kdc[21531](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126351, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.18.154.62@KRBTEST.COM
I20260807 18:12:31.389778 22413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:31.374334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58856 (local address 127.18.154.62:43221)
0807 18:12:31.374973 (+ 639us) server_negotiation.cc:206] Beginning negotiation
0807 18:12:31.374978 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:12:31.375001 (+ 23us) server_negotiation.cc:407] Connection header received
0807 18:12:31.375158 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:12:31.375163 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:12:31.375229 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:12:31.375326 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:12:31.376774 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:31.377725 (+ 951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:31.379087 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:31.379554 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:31.379681 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:31.383933 (+ 4252us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:12:31.383939 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:12:31.383942 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:12:31.383980 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:12:31.387091 (+ 3111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:12:31.387741 (+ 650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:12:31.387747 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:12:31.387751 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:12:31.387888 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:12:31.388280 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:12:31.388286 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:12:31.388291 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:12:31.388570 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:12:31.388842 (+ 272us) server_negotiation.cc:1035] Waiting for connection context
0807 18:12:31.389131 (+ 289us) server_negotiation.cc:299] Negotiation successful
0807 18:12:31.389363 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":518,"thread_start_us":342,"threads_started":1}
I20260807 18:12:31.393875 21592 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58856:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 18:12:31.394546 21592 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:12:31.402201 21784 tablet_service.cc:1467] Processing CreateTablet for tablet ed31db3cefb44eb6946f5619cbd1b64f (DEFAULT_TABLE table=test-table [id=b843f3b4bb024b11a4c768e305c77598]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:12:31.402288 21967 tablet_service.cc:1467] Processing CreateTablet for tablet ed31db3cefb44eb6946f5619cbd1b64f (DEFAULT_TABLE table=test-table [id=b843f3b4bb024b11a4c768e305c77598]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:12:31.402559 22160 tablet_service.cc:1467] Processing CreateTablet for tablet ed31db3cefb44eb6946f5619cbd1b64f (DEFAULT_TABLE table=test-table [id=b843f3b4bb024b11a4c768e305c77598]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:12:31.402822 21784 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ed31db3cefb44eb6946f5619cbd1b64f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:12:31.402822 21967 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ed31db3cefb44eb6946f5619cbd1b64f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:12:31.403051 22160 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ed31db3cefb44eb6946f5619cbd1b64f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:12:31.408406 22414 tablet_bootstrap.cc:492] T ed31db3cefb44eb6946f5619cbd1b64f P f7ddbaeafa194da88d61994cf63503f6: Bootstrap starting.
I20260807 18:12:31.410331 22415 tablet_bootstrap.cc:492] T ed31db3cefb44eb6946f5619cbd1b64f P bf17273192884a1e8a41b74403970a98: Bootstrap starting.
I20260807 18:12:31.410624 22416 tablet_bootstrap.cc:492] T ed31db3cefb44eb6946f5619cbd1b64f P f5c386f2052b4c86a1c451abcc9ca25e: Bootstrap starting.
I20260807 18:12:31.412145 22414 tablet_bootstrap.cc:654] T ed31db3cefb44eb6946f5619cbd1b64f P f7ddbaeafa194da88d61994cf63503f6: Neither blocks nor log segments found. Creating new log.
I20260807 18:12:31.412266 22415 tablet_bootstrap.cc:654] T ed31db3cefb44eb6946f5619cbd1b64f P bf17273192884a1e8a41b74403970a98: Neither blocks nor log segments found. Creating new log.
I20260807 18:12:31.412339 22416 tablet_bootstrap.cc:654] T ed31db3cefb44eb6946f5619cbd1b64f P f5c386f2052b4c86a1c451abcc9ca25e: Neither blocks nor log segments found. Creating new log.
I20260807 18:12:31.415297 22414 tablet_bootstrap.cc:492] T ed31db3cefb44eb6946f5619cbd1b64f P f7ddbaeafa194da88d61994cf63503f6: No bootstrap required, opened a new log
I20260807 18:12:31.415422 22415 tablet_bootstrap.cc:492] T ed31db3cefb44eb6946f5619cbd1b64f P bf17273192884a1e8a41b74403970a98: No bootstrap required, opened a new log
I20260807 18:12:31.415506 22414 ts_tablet_manager.cc:1397] T ed31db3cefb44eb6946f5619cbd1b64f P f7ddbaeafa194da88d61994cf63503f6: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260807 18:12:31.415580 22415 ts_tablet_manager.cc:1397] T ed31db3cefb44eb6946f5619cbd1b64f P bf17273192884a1e8a41b74403970a98: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260807 18:12:31.416250 22415 raft_consensus.cc:348] T ed31db3cefb44eb6946f5619cbd1b64f P bf17273192884a1e8a41b74403970a98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7ddbaeafa194da88d61994cf63503f6" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 38089 } } peers { permanent_uuid: "bf17273192884a1e8a41b74403970a98" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 39793 } } peers { permanent_uuid: "f5c386f2052b4c86a1c451abcc9ca25e" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 43031 } }
I20260807 18:12:31.416507 22415 raft_consensus.cc:374] T ed31db3cefb44eb6946f5619cbd1b64f P bf17273192884a1e8a41b74403970a98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:12:31.416618 22415 raft_consensus.cc:729] T ed31db3cefb44eb6946f5619cbd1b64f P bf17273192884a1e8a41b74403970a98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf17273192884a1e8a41b74403970a98, State: Initialized, Role: FOLLOWER
I20260807 18:12:31.416653 22414 raft_consensus.cc:348] T ed31db3cefb44eb6946f5619cbd1b64f P f7ddbaeafa194da88d61994cf63503f6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7ddbaeafa194da88d61994cf63503f6" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 38089 } } peers { permanent_uuid: "bf17273192884a1e8a41b74403970a98" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 39793 } } peers { permanent_uuid: "f5c386f2052b4c86a1c451abcc9ca25e" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 43031 } }
I20260807 18:12:31.417001 22414 raft_consensus.cc:374] T ed31db3cefb44eb6946f5619cbd1b64f P f7ddbaeafa194da88d61994cf63503f6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:12:31.416909 22415 consensus_queue.cc:260] T ed31db3cefb44eb6946f5619cbd1b64f P bf17273192884a1e8a41b74403970a98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7ddbaeafa194da88d61994cf63503f6" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 38089 } } peers { permanent_uuid: "bf17273192884a1e8a41b74403970a98" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 39793 } } peers { permanent_uuid: "f5c386f2052b4c86a1c451abcc9ca25e" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 43031 } }
I20260807 18:12:31.417121 22414 raft_consensus.cc:729] T ed31db3cefb44eb6946f5619cbd1b64f P f7ddbaeafa194da88d61994cf63503f6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7ddbaeafa194da88d61994cf63503f6, State: Initialized, Role: FOLLOWER
I20260807 18:12:31.417421 22414 consensus_queue.cc:260] T ed31db3cefb44eb6946f5619cbd1b64f P f7ddbaeafa194da88d61994cf63503f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7ddbaeafa194da88d61994cf63503f6" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 38089 } } peers { permanent_uuid: "bf17273192884a1e8a41b74403970a98" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 39793 } } peers { permanent_uuid: "f5c386f2052b4c86a1c451abcc9ca25e" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 43031 } }
I20260807 18:12:31.419770 22415 ts_tablet_manager.cc:1428] T ed31db3cefb44eb6946f5619cbd1b64f P bf17273192884a1e8a41b74403970a98: Time spent starting tablet: real 0.004s user 0.000s sys 0.004s
I20260807 18:12:31.420413 22414 ts_tablet_manager.cc:1428] T ed31db3cefb44eb6946f5619cbd1b64f P f7ddbaeafa194da88d61994cf63503f6: Time spent starting tablet: real 0.005s user 0.003s sys 0.000s
I20260807 18:12:31.420989 22416 tablet_bootstrap.cc:492] T ed31db3cefb44eb6946f5619cbd1b64f P f5c386f2052b4c86a1c451abcc9ca25e: No bootstrap required, opened a new log
I20260807 18:12:31.421173 22416 ts_tablet_manager.cc:1397] T ed31db3cefb44eb6946f5619cbd1b64f P f5c386f2052b4c86a1c451abcc9ca25e: Time spent bootstrapping tablet: real 0.011s user 0.000s sys 0.009s
I20260807 18:12:31.422060 22416 raft_consensus.cc:348] T ed31db3cefb44eb6946f5619cbd1b64f P f5c386f2052b4c86a1c451abcc9ca25e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7ddbaeafa194da88d61994cf63503f6" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 38089 } } peers { permanent_uuid: "bf17273192884a1e8a41b74403970a98" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 39793 } } peers { permanent_uuid: "f5c386f2052b4c86a1c451abcc9ca25e" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 43031 } }
I20260807 18:12:31.422297 22416 raft_consensus.cc:374] T ed31db3cefb44eb6946f5619cbd1b64f P f5c386f2052b4c86a1c451abcc9ca25e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:12:31.422405 22416 raft_consensus.cc:729] T ed31db3cefb44eb6946f5619cbd1b64f P f5c386f2052b4c86a1c451abcc9ca25e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5c386f2052b4c86a1c451abcc9ca25e, State: Initialized, Role: FOLLOWER
I20260807 18:12:31.422688 22416 consensus_queue.cc:260] T ed31db3cefb44eb6946f5619cbd1b64f P f5c386f2052b4c86a1c451abcc9ca25e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7ddbaeafa194da88d61994cf63503f6" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 38089 } } peers { permanent_uuid: "bf17273192884a1e8a41b74403970a98" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 39793 } } peers { permanent_uuid: "f5c386f2052b4c86a1c451abcc9ca25e" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 43031 } }
I20260807 18:12:31.423329 22416 ts_tablet_manager.cc:1428] T ed31db3cefb44eb6946f5619cbd1b64f P f5c386f2052b4c86a1c451abcc9ca25e: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260807 18:12:31.541492 22389 raft_consensus.cc:482] T ed31db3cefb44eb6946f5619cbd1b64f P f5c386f2052b4c86a1c451abcc9ca25e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:12:31.541769 22389 raft_consensus.cc:504] T ed31db3cefb44eb6946f5619cbd1b64f P f5c386f2052b4c86a1c451abcc9ca25e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7ddbaeafa194da88d61994cf63503f6" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 38089 } } peers { permanent_uuid: "bf17273192884a1e8a41b74403970a98" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 39793 } } peers { permanent_uuid: "f5c386f2052b4c86a1c451abcc9ca25e" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 43031 } }
I20260807 18:12:31.542500 22389 leader_election.cc:288] T ed31db3cefb44eb6946f5619cbd1b64f P f5c386f2052b4c86a1c451abcc9ca25e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f7ddbaeafa194da88d61994cf63503f6 (127.18.154.1:38089), bf17273192884a1e8a41b74403970a98 (127.18.154.2:39793)
I20260807 18:12:31.543272 22000 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed31db3cefb44eb6946f5619cbd1b64f" candidate_uuid: "f5c386f2052b4c86a1c451abcc9ca25e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf17273192884a1e8a41b74403970a98" is_pre_election: true
I20260807 18:12:31.543252 21814 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed31db3cefb44eb6946f5619cbd1b64f" candidate_uuid: "f5c386f2052b4c86a1c451abcc9ca25e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7ddbaeafa194da88d61994cf63503f6" is_pre_election: true
I20260807 18:12:31.543583 22000 raft_consensus.cc:2454] T ed31db3cefb44eb6946f5619cbd1b64f P bf17273192884a1e8a41b74403970a98 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5c386f2052b4c86a1c451abcc9ca25e in term 0.
I20260807 18:12:31.543601 21814 raft_consensus.cc:2454] T ed31db3cefb44eb6946f5619cbd1b64f P f7ddbaeafa194da88d61994cf63503f6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5c386f2052b4c86a1c451abcc9ca25e in term 0.
I20260807 18:12:31.544179 22095 leader_election.cc:302] T ed31db3cefb44eb6946f5619cbd1b64f P f5c386f2052b4c86a1c451abcc9ca25e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: bf17273192884a1e8a41b74403970a98, f5c386f2052b4c86a1c451abcc9ca25e, f7ddbaeafa194da88d61994cf63503f6; no voters:
I20260807 18:12:31.544485 22389 raft_consensus.cc:2781] T ed31db3cefb44eb6946f5619cbd1b64f P f5c386f2052b4c86a1c451abcc9ca25e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:12:31.544620 22389 raft_consensus.cc:482] T ed31db3cefb44eb6946f5619cbd1b64f P f5c386f2052b4c86a1c451abcc9ca25e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:12:31.544749 22389 raft_consensus.cc:3037] T ed31db3cefb44eb6946f5619cbd1b64f P f5c386f2052b4c86a1c451abcc9ca25e [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:12:31.547180 22389 raft_consensus.cc:504] T ed31db3cefb44eb6946f5619cbd1b64f P f5c386f2052b4c86a1c451abcc9ca25e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7ddbaeafa194da88d61994cf63503f6" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 38089 } } peers { permanent_uuid: "bf17273192884a1e8a41b74403970a98" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 39793 } } peers { permanent_uuid: "f5c386f2052b4c86a1c451abcc9ca25e" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 43031 } }
I20260807 18:12:31.547824 22389 leader_election.cc:288] T ed31db3cefb44eb6946f5619cbd1b64f P f5c386f2052b4c86a1c451abcc9ca25e [CANDIDATE]: Term 1 election: Requested vote from peers f7ddbaeafa194da88d61994cf63503f6 (127.18.154.1:38089), bf17273192884a1e8a41b74403970a98 (127.18.154.2:39793)
I20260807 18:12:31.548434 21814 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed31db3cefb44eb6946f5619cbd1b64f" candidate_uuid: "f5c386f2052b4c86a1c451abcc9ca25e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7ddbaeafa194da88d61994cf63503f6"
I20260807 18:12:31.548473 22000 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed31db3cefb44eb6946f5619cbd1b64f" candidate_uuid: "f5c386f2052b4c86a1c451abcc9ca25e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf17273192884a1e8a41b74403970a98"
I20260807 18:12:31.548664 21814 raft_consensus.cc:3037] T ed31db3cefb44eb6946f5619cbd1b64f P f7ddbaeafa194da88d61994cf63503f6 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:12:31.548719 22000 raft_consensus.cc:3037] T ed31db3cefb44eb6946f5619cbd1b64f P bf17273192884a1e8a41b74403970a98 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:12:31.550881 21814 raft_consensus.cc:2454] T ed31db3cefb44eb6946f5619cbd1b64f P f7ddbaeafa194da88d61994cf63503f6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5c386f2052b4c86a1c451abcc9ca25e in term 1.
I20260807 18:12:31.551156 22000 raft_consensus.cc:2454] T ed31db3cefb44eb6946f5619cbd1b64f P bf17273192884a1e8a41b74403970a98 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5c386f2052b4c86a1c451abcc9ca25e in term 1.
I20260807 18:12:31.551362 22097 leader_election.cc:302] T ed31db3cefb44eb6946f5619cbd1b64f P f5c386f2052b4c86a1c451abcc9ca25e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f5c386f2052b4c86a1c451abcc9ca25e, f7ddbaeafa194da88d61994cf63503f6; no voters:
I20260807 18:12:31.551707 22389 raft_consensus.cc:2781] T ed31db3cefb44eb6946f5619cbd1b64f P f5c386f2052b4c86a1c451abcc9ca25e [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:12:31.551883 22389 raft_consensus.cc:686] T ed31db3cefb44eb6946f5619cbd1b64f P f5c386f2052b4c86a1c451abcc9ca25e [term 1 LEADER]: Becoming Leader. State: Replica: f5c386f2052b4c86a1c451abcc9ca25e, State: Running, Role: LEADER
I20260807 18:12:31.552198 22389 consensus_queue.cc:237] T ed31db3cefb44eb6946f5619cbd1b64f P f5c386f2052b4c86a1c451abcc9ca25e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7ddbaeafa194da88d61994cf63503f6" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 38089 } } peers { permanent_uuid: "bf17273192884a1e8a41b74403970a98" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 39793 } } peers { permanent_uuid: "f5c386f2052b4c86a1c451abcc9ca25e" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 43031 } }
I20260807 18:12:31.556348 21594 catalog_manager.cc:5495] T ed31db3cefb44eb6946f5619cbd1b64f P f5c386f2052b4c86a1c451abcc9ca25e reported cstate change: term changed from 0 to 1, leader changed from <none> to f5c386f2052b4c86a1c451abcc9ca25e (127.18.154.3). New cstate: current_term: 1 leader_uuid: "f5c386f2052b4c86a1c451abcc9ca25e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7ddbaeafa194da88d61994cf63503f6" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 38089 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bf17273192884a1e8a41b74403970a98" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 39793 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5c386f2052b4c86a1c451abcc9ca25e" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 43031 } health_report { overall_health: HEALTHY } } }
I20260807 18:12:31.585811 22426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:31.580244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45094 (local address 127.18.154.3:43031)
0807 18:12:31.580868 (+ 624us) server_negotiation.cc:206] Beginning negotiation
0807 18:12:31.580873 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:12:31.581189 (+ 316us) server_negotiation.cc:407] Connection header received
0807 18:12:31.581244 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:12:31.581248 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:12:31.581313 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:12:31.581395 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:12:31.581760 (+ 365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:31.582577 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:31.584002 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:31.584494 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:31.584589 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:31.585046 (+ 457us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:12:31.585150 (+ 104us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:12:31.585218 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0807 18:12:31.585351 (+ 133us) server_negotiation.cc:299] Negotiation successful
0807 18:12:31.585412 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":520,"thread_start_us":392,"threads_started":1}
I20260807 18:12:31.595999 22000 raft_consensus.cc:1261] T ed31db3cefb44eb6946f5619cbd1b64f P bf17273192884a1e8a41b74403970a98 [term 1 FOLLOWER]: Refusing update from remote peer f5c386f2052b4c86a1c451abcc9ca25e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:12:31.595978 21814 raft_consensus.cc:1261] T ed31db3cefb44eb6946f5619cbd1b64f P f7ddbaeafa194da88d61994cf63503f6 [term 1 FOLLOWER]: Refusing update from remote peer f5c386f2052b4c86a1c451abcc9ca25e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:12:31.596580 22389 consensus_queue.cc:1035] T ed31db3cefb44eb6946f5619cbd1b64f P f5c386f2052b4c86a1c451abcc9ca25e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bf17273192884a1e8a41b74403970a98" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 39793 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:12:31.596921 22322 consensus_queue.cc:1035] T ed31db3cefb44eb6946f5619cbd1b64f P f5c386f2052b4c86a1c451abcc9ca25e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f7ddbaeafa194da88d61994cf63503f6" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 38089 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:12:31.646852 21593 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58856:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:12:31.647183 21593 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58856:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:12:31.650534 21593 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6fff555ae1b94e1494e4fb470dad5674: Sending DeleteTablet for 3 replicas of tablet ed31db3cefb44eb6946f5619cbd1b64f
I20260807 18:12:31.651448 21784 tablet_service.cc:1514] Processing DeleteTablet for tablet ed31db3cefb44eb6946f5619cbd1b64f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:12:31 UTC) from {username='kudu', principal='kudu/127.18.154.62@KRBTEST.COM'} at 127.0.0.1:55412
I20260807 18:12:31.651595 21967 tablet_service.cc:1514] Processing DeleteTablet for tablet ed31db3cefb44eb6946f5619cbd1b64f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:12:31 UTC) from {username='kudu', principal='kudu/127.18.154.62@KRBTEST.COM'} at 127.0.0.1:36944
I20260807 18:12:31.651912 22402 tablet_replica.cc:330] T ed31db3cefb44eb6946f5619cbd1b64f P bf17273192884a1e8a41b74403970a98: stopping tablet replica
I20260807 18:12:31.652086 22402 raft_consensus.cc:2229] T ed31db3cefb44eb6946f5619cbd1b64f P bf17273192884a1e8a41b74403970a98 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:12:31.652251 22402 raft_consensus.cc:2258] T ed31db3cefb44eb6946f5619cbd1b64f P bf17273192884a1e8a41b74403970a98 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:12:31.652561 22160 tablet_service.cc:1514] Processing DeleteTablet for tablet ed31db3cefb44eb6946f5619cbd1b64f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:12:31 UTC) from {username='kudu', principal='kudu/127.18.154.62@KRBTEST.COM'} at 127.0.0.1:45078
I20260807 18:12:31.652878 22404 tablet_replica.cc:330] T ed31db3cefb44eb6946f5619cbd1b64f P f5c386f2052b4c86a1c451abcc9ca25e: stopping tablet replica
I20260807 18:12:31.653084 22404 raft_consensus.cc:2229] T ed31db3cefb44eb6946f5619cbd1b64f P f5c386f2052b4c86a1c451abcc9ca25e [term 1 LEADER]: Raft consensus shutting down.
I20260807 18:12:31.653403 22404 raft_consensus.cc:2258] T ed31db3cefb44eb6946f5619cbd1b64f P f5c386f2052b4c86a1c451abcc9ca25e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:12:31.654006 22402 ts_tablet_manager.cc:1905] T ed31db3cefb44eb6946f5619cbd1b64f P bf17273192884a1e8a41b74403970a98: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:12:31.654104 22403 tablet_replica.cc:330] T ed31db3cefb44eb6946f5619cbd1b64f P f7ddbaeafa194da88d61994cf63503f6: stopping tablet replica
I20260807 18:12:31.654371 22404 ts_tablet_manager.cc:1905] T ed31db3cefb44eb6946f5619cbd1b64f P f5c386f2052b4c86a1c451abcc9ca25e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:12:31.654340 22403 raft_consensus.cc:2229] T ed31db3cefb44eb6946f5619cbd1b64f P f7ddbaeafa194da88d61994cf63503f6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:12:31.654564 22403 raft_consensus.cc:2258] T ed31db3cefb44eb6946f5619cbd1b64f P f7ddbaeafa194da88d61994cf63503f6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:12:31.657788 21593 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58856:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 18:12:31.658679 21593 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:12:31.661921 22402 ts_tablet_manager.cc:1918] T ed31db3cefb44eb6946f5619cbd1b64f P bf17273192884a1e8a41b74403970a98: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 18:12:31.662102 22402 log.cc:1196] T ed31db3cefb44eb6946f5619cbd1b64f P bf17273192884a1e8a41b74403970a98: Deleting WAL directory at /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126328808105-19048-0/minicluster-data/ts-1/wal/wals/ed31db3cefb44eb6946f5619cbd1b64f
I20260807 18:12:31.662338 22404 ts_tablet_manager.cc:1918] T ed31db3cefb44eb6946f5619cbd1b64f P f5c386f2052b4c86a1c451abcc9ca25e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 18:12:31.662591 22402 ts_tablet_manager.cc:1939] T ed31db3cefb44eb6946f5619cbd1b64f P bf17273192884a1e8a41b74403970a98: Deleting consensus metadata
I20260807 18:12:31.663743 21572 catalog_manager.cc:4841] TS bf17273192884a1e8a41b74403970a98 (127.18.154.2:39793): tablet ed31db3cefb44eb6946f5619cbd1b64f (table test-table [id=b843f3b4bb024b11a4c768e305c77598]) successfully deleted
I20260807 18:12:31.664392 22404 log.cc:1196] T ed31db3cefb44eb6946f5619cbd1b64f P f5c386f2052b4c86a1c451abcc9ca25e: Deleting WAL directory at /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126328808105-19048-0/minicluster-data/ts-2/wal/wals/ed31db3cefb44eb6946f5619cbd1b64f
I20260807 18:12:31.664829 22404 ts_tablet_manager.cc:1939] T ed31db3cefb44eb6946f5619cbd1b64f P f5c386f2052b4c86a1c451abcc9ca25e: Deleting consensus metadata
I20260807 18:12:31.666276 21571 catalog_manager.cc:4841] TS f5c386f2052b4c86a1c451abcc9ca25e (127.18.154.3:43031): tablet ed31db3cefb44eb6946f5619cbd1b64f (table test-table [id=b843f3b4bb024b11a4c768e305c77598]) successfully deleted
I20260807 18:12:31.666744 22403 ts_tablet_manager.cc:1905] T ed31db3cefb44eb6946f5619cbd1b64f P f7ddbaeafa194da88d61994cf63503f6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:12:31.667313 21784 tablet_service.cc:1467] Processing CreateTablet for tablet 2c8fbebc008e4d1995d2065b893b01dc (DEFAULT_TABLE table=test-table [id=fd81d65f77864250abd5c398d546fb29]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:12:31.667569 21967 tablet_service.cc:1467] Processing CreateTablet for tablet 2c8fbebc008e4d1995d2065b893b01dc (DEFAULT_TABLE table=test-table [id=fd81d65f77864250abd5c398d546fb29]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:12:31.667771 21784 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2c8fbebc008e4d1995d2065b893b01dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:12:31.667963 22160 tablet_service.cc:1467] Processing CreateTablet for tablet 2c8fbebc008e4d1995d2065b893b01dc (DEFAULT_TABLE table=test-table [id=fd81d65f77864250abd5c398d546fb29]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:12:31.668071 21967 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2c8fbebc008e4d1995d2065b893b01dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:12:31.668361 22160 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2c8fbebc008e4d1995d2065b893b01dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:12:31.674656 22403 ts_tablet_manager.cc:1918] T ed31db3cefb44eb6946f5619cbd1b64f P f7ddbaeafa194da88d61994cf63503f6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 18:12:31.676008 22403 log.cc:1196] T ed31db3cefb44eb6946f5619cbd1b64f P f7ddbaeafa194da88d61994cf63503f6: Deleting WAL directory at /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126328808105-19048-0/minicluster-data/ts-0/wal/wals/ed31db3cefb44eb6946f5619cbd1b64f
I20260807 18:12:31.676043 22416 tablet_bootstrap.cc:492] T 2c8fbebc008e4d1995d2065b893b01dc P f5c386f2052b4c86a1c451abcc9ca25e: Bootstrap starting.
I20260807 18:12:31.676542 22403 ts_tablet_manager.cc:1939] T ed31db3cefb44eb6946f5619cbd1b64f P f7ddbaeafa194da88d61994cf63503f6: Deleting consensus metadata
I20260807 18:12:31.677378 22415 tablet_bootstrap.cc:492] T 2c8fbebc008e4d1995d2065b893b01dc P bf17273192884a1e8a41b74403970a98: Bootstrap starting.
I20260807 18:12:31.677723 21575 catalog_manager.cc:4841] TS f7ddbaeafa194da88d61994cf63503f6 (127.18.154.1:38089): tablet ed31db3cefb44eb6946f5619cbd1b64f (table test-table [id=b843f3b4bb024b11a4c768e305c77598]) successfully deleted
I20260807 18:12:31.678188 22416 tablet_bootstrap.cc:654] T 2c8fbebc008e4d1995d2065b893b01dc P f5c386f2052b4c86a1c451abcc9ca25e: Neither blocks nor log segments found. Creating new log.
I20260807 18:12:31.679615 22415 tablet_bootstrap.cc:654] T 2c8fbebc008e4d1995d2065b893b01dc P bf17273192884a1e8a41b74403970a98: Neither blocks nor log segments found. Creating new log.
I20260807 18:12:31.680743 22414 tablet_bootstrap.cc:492] T 2c8fbebc008e4d1995d2065b893b01dc P f7ddbaeafa194da88d61994cf63503f6: Bootstrap starting.
I20260807 18:12:31.683501 22414 tablet_bootstrap.cc:654] T 2c8fbebc008e4d1995d2065b893b01dc P f7ddbaeafa194da88d61994cf63503f6: Neither blocks nor log segments found. Creating new log.
I20260807 18:12:31.687500 22416 tablet_bootstrap.cc:492] T 2c8fbebc008e4d1995d2065b893b01dc P f5c386f2052b4c86a1c451abcc9ca25e: No bootstrap required, opened a new log
I20260807 18:12:31.687687 22416 ts_tablet_manager.cc:1397] T 2c8fbebc008e4d1995d2065b893b01dc P f5c386f2052b4c86a1c451abcc9ca25e: Time spent bootstrapping tablet: real 0.012s user 0.008s sys 0.002s
I20260807 18:12:31.688230 22415 tablet_bootstrap.cc:492] T 2c8fbebc008e4d1995d2065b893b01dc P bf17273192884a1e8a41b74403970a98: No bootstrap required, opened a new log
I20260807 18:12:31.688453 22415 ts_tablet_manager.cc:1397] T 2c8fbebc008e4d1995d2065b893b01dc P bf17273192884a1e8a41b74403970a98: Time spent bootstrapping tablet: real 0.011s user 0.006s sys 0.002s
I20260807 18:12:31.688468 22416 raft_consensus.cc:348] T 2c8fbebc008e4d1995d2065b893b01dc P f5c386f2052b4c86a1c451abcc9ca25e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7ddbaeafa194da88d61994cf63503f6" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 38089 } } peers { permanent_uuid: "bf17273192884a1e8a41b74403970a98" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 39793 } } peers { permanent_uuid: "f5c386f2052b4c86a1c451abcc9ca25e" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 43031 } }
I20260807 18:12:31.688740 22416 raft_consensus.cc:374] T 2c8fbebc008e4d1995d2065b893b01dc P f5c386f2052b4c86a1c451abcc9ca25e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:12:31.688817 22416 raft_consensus.cc:729] T 2c8fbebc008e4d1995d2065b893b01dc P f5c386f2052b4c86a1c451abcc9ca25e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5c386f2052b4c86a1c451abcc9ca25e, State: Initialized, Role: FOLLOWER
I20260807 18:12:31.689038 22416 consensus_queue.cc:260] T 2c8fbebc008e4d1995d2065b893b01dc P f5c386f2052b4c86a1c451abcc9ca25e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7ddbaeafa194da88d61994cf63503f6" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 38089 } } peers { permanent_uuid: "bf17273192884a1e8a41b74403970a98" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 39793 } } peers { permanent_uuid: "f5c386f2052b4c86a1c451abcc9ca25e" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 43031 } }
I20260807 18:12:31.689196 22415 raft_consensus.cc:348] T 2c8fbebc008e4d1995d2065b893b01dc P bf17273192884a1e8a41b74403970a98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7ddbaeafa194da88d61994cf63503f6" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 38089 } } peers { permanent_uuid: "bf17273192884a1e8a41b74403970a98" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 39793 } } peers { permanent_uuid: "f5c386f2052b4c86a1c451abcc9ca25e" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 43031 } }
I20260807 18:12:31.689558 22416 ts_tablet_manager.cc:1428] T 2c8fbebc008e4d1995d2065b893b01dc P f5c386f2052b4c86a1c451abcc9ca25e: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260807 18:12:31.689489 22415 raft_consensus.cc:374] T 2c8fbebc008e4d1995d2065b893b01dc P bf17273192884a1e8a41b74403970a98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:12:31.690594 22415 raft_consensus.cc:729] T 2c8fbebc008e4d1995d2065b893b01dc P bf17273192884a1e8a41b74403970a98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf17273192884a1e8a41b74403970a98, State: Initialized, Role: FOLLOWER
I20260807 18:12:31.690848 22414 tablet_bootstrap.cc:492] T 2c8fbebc008e4d1995d2065b893b01dc P f7ddbaeafa194da88d61994cf63503f6: No bootstrap required, opened a new log
I20260807 18:12:31.691043 22414 ts_tablet_manager.cc:1397] T 2c8fbebc008e4d1995d2065b893b01dc P f7ddbaeafa194da88d61994cf63503f6: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.000s
I20260807 18:12:31.691768 22414 raft_consensus.cc:348] T 2c8fbebc008e4d1995d2065b893b01dc P f7ddbaeafa194da88d61994cf63503f6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7ddbaeafa194da88d61994cf63503f6" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 38089 } } peers { permanent_uuid: "bf17273192884a1e8a41b74403970a98" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 39793 } } peers { permanent_uuid: "f5c386f2052b4c86a1c451abcc9ca25e" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 43031 } }
I20260807 18:12:31.692056 22414 raft_consensus.cc:374] T 2c8fbebc008e4d1995d2065b893b01dc P f7ddbaeafa194da88d61994cf63503f6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:12:31.692158 22414 raft_consensus.cc:729] T 2c8fbebc008e4d1995d2065b893b01dc P f7ddbaeafa194da88d61994cf63503f6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7ddbaeafa194da88d61994cf63503f6, State: Initialized, Role: FOLLOWER
I20260807 18:12:31.692428 22414 consensus_queue.cc:260] T 2c8fbebc008e4d1995d2065b893b01dc P f7ddbaeafa194da88d61994cf63503f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7ddbaeafa194da88d61994cf63503f6" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 38089 } } peers { permanent_uuid: "bf17273192884a1e8a41b74403970a98" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 39793 } } peers { permanent_uuid: "f5c386f2052b4c86a1c451abcc9ca25e" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 43031 } }
I20260807 18:12:31.692929 22414 ts_tablet_manager.cc:1428] T 2c8fbebc008e4d1995d2065b893b01dc P f7ddbaeafa194da88d61994cf63503f6: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260807 18:12:31.693166 22415 consensus_queue.cc:260] T 2c8fbebc008e4d1995d2065b893b01dc P bf17273192884a1e8a41b74403970a98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7ddbaeafa194da88d61994cf63503f6" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 38089 } } peers { permanent_uuid: "bf17273192884a1e8a41b74403970a98" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 39793 } } peers { permanent_uuid: "f5c386f2052b4c86a1c451abcc9ca25e" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 43031 } }
I20260807 18:12:31.693804 22415 ts_tablet_manager.cc:1428] T 2c8fbebc008e4d1995d2065b893b01dc P bf17273192884a1e8a41b74403970a98: Time spent starting tablet: real 0.005s user 0.003s sys 0.000s
I20260807 18:12:31.846591 22420 raft_consensus.cc:482] T 2c8fbebc008e4d1995d2065b893b01dc P bf17273192884a1e8a41b74403970a98 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:12:31.846951 22420 raft_consensus.cc:504] T 2c8fbebc008e4d1995d2065b893b01dc P bf17273192884a1e8a41b74403970a98 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7ddbaeafa194da88d61994cf63503f6" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 38089 } } peers { permanent_uuid: "bf17273192884a1e8a41b74403970a98" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 39793 } } peers { permanent_uuid: "f5c386f2052b4c86a1c451abcc9ca25e" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 43031 } }
I20260807 18:12:31.848891 22420 leader_election.cc:288] T 2c8fbebc008e4d1995d2065b893b01dc P bf17273192884a1e8a41b74403970a98 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f7ddbaeafa194da88d61994cf63503f6 (127.18.154.1:38089), f5c386f2052b4c86a1c451abcc9ca25e (127.18.154.3:43031)
I20260807 18:12:31.872653 22426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:31.850019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.2:46155 (local address 127.18.154.3:43031)
0807 18:12:31.850215 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0807 18:12:31.850222 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:12:31.863772 (+ 13550us) server_negotiation.cc:407] Connection header received
0807 18:12:31.863980 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:12:31.863986 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:12:31.864066 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:12:31.864173 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:12:31.865176 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:31.866371 (+ 1195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:31.869718 (+ 3347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:31.871148 (+ 1430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:31.871323 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:31.871405 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0807 18:12:31.872220 (+ 815us) server_negotiation.cc:299] Negotiation successful
0807 18:12:31.872362 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260807 18:12:31.873139 22436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:31.849316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.3:43031 (local address 127.18.154.2:46155)
0807 18:12:31.863614 (+ 14298us) negotiation.cc:107] Waiting for socket to connect
0807 18:12:31.863633 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 18:12:31.863811 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:12:31.864223 (+ 412us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:12:31.864230 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:12:31.864264 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:12:31.864887 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:12:31.864906 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:12:31.866536 (+ 1630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:12:31.866543 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:12:31.869512 (+ 2969us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:12:31.869564 (+ 52us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:12:31.871545 (+ 1981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:12:31.871552 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:12:31.871849 (+ 297us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:31.872011 (+ 162us) client_negotiation.cc:769] Sending connection context
0807 18:12:31.872117 (+ 106us) client_negotiation.cc:240] Negotiation successful
0807 18:12:31.872284 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14152,"mutex_wait_us":78,"spinlock_wait_cycles":896,"thread_start_us":12479,"threads_started":1}
I20260807 18:12:31.873322 22437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:31.849470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.2:59621 (local address 127.18.154.1:38089)
0807 18:12:31.862988 (+ 13518us) server_negotiation.cc:206] Beginning negotiation
0807 18:12:31.862995 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:12:31.863024 (+ 29us) server_negotiation.cc:407] Connection header received
0807 18:12:31.863101 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:12:31.863106 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:12:31.863260 (+ 154us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:12:31.863370 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:12:31.864385 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:31.865732 (+ 1347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:31.869720 (+ 3988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:31.871227 (+ 1507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:31.871426 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:31.871498 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0807 18:12:31.872723 (+ 1225us) server_negotiation.cc:299] Negotiation successful
0807 18:12:31.872943 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13357,"thread_start_us":12863,"threads_started":1}
I20260807 18:12:31.874074 22438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:31.849155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.1:38089 (local address 127.18.154.2:59621)
0807 18:12:31.862219 (+ 13064us) negotiation.cc:107] Waiting for socket to connect
0807 18:12:31.862251 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0807 18:12:31.862490 (+ 239us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:12:31.863561 (+ 1071us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:12:31.863569 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:12:31.863607 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:12:31.864119 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:12:31.864163 (+ 44us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:12:31.865871 (+ 1708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:12:31.865879 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:12:31.869173 (+ 3294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:12:31.869201 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:12:31.871750 (+ 2549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:12:31.871758 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:12:31.872042 (+ 284us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:31.872104 (+ 62us) client_negotiation.cc:769] Sending connection context
0807 18:12:31.872217 (+ 113us) client_negotiation.cc:240] Negotiation successful
0807 18:12:31.872362 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12860,"mutex_wait_us":52,"spinlock_wait_cycles":15488,"thread_start_us":14097,"threads_started":1}
I20260807 18:12:31.874045 22187 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c8fbebc008e4d1995d2065b893b01dc" candidate_uuid: "bf17273192884a1e8a41b74403970a98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5c386f2052b4c86a1c451abcc9ca25e" is_pre_election: true
I20260807 18:12:31.874392 22187 raft_consensus.cc:2454] T 2c8fbebc008e4d1995d2065b893b01dc P f5c386f2052b4c86a1c451abcc9ca25e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bf17273192884a1e8a41b74403970a98 in term 0.
I20260807 18:12:31.874908 21814 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c8fbebc008e4d1995d2065b893b01dc" candidate_uuid: "bf17273192884a1e8a41b74403970a98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7ddbaeafa194da88d61994cf63503f6" is_pre_election: true
I20260807 18:12:31.875047 21904 leader_election.cc:302] T 2c8fbebc008e4d1995d2065b893b01dc P bf17273192884a1e8a41b74403970a98 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bf17273192884a1e8a41b74403970a98, f5c386f2052b4c86a1c451abcc9ca25e; no voters:
I20260807 18:12:31.875175 21814 raft_consensus.cc:2454] T 2c8fbebc008e4d1995d2065b893b01dc P f7ddbaeafa194da88d61994cf63503f6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bf17273192884a1e8a41b74403970a98 in term 0.
I20260807 18:12:31.875486 22420 raft_consensus.cc:2781] T 2c8fbebc008e4d1995d2065b893b01dc P bf17273192884a1e8a41b74403970a98 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:12:31.875619 22420 raft_consensus.cc:482] T 2c8fbebc008e4d1995d2065b893b01dc P bf17273192884a1e8a41b74403970a98 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:12:31.875761 22420 raft_consensus.cc:3037] T 2c8fbebc008e4d1995d2065b893b01dc P bf17273192884a1e8a41b74403970a98 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:12:31.878619 22420 raft_consensus.cc:504] T 2c8fbebc008e4d1995d2065b893b01dc P bf17273192884a1e8a41b74403970a98 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7ddbaeafa194da88d61994cf63503f6" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 38089 } } peers { permanent_uuid: "bf17273192884a1e8a41b74403970a98" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 39793 } } peers { permanent_uuid: "f5c386f2052b4c86a1c451abcc9ca25e" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 43031 } }
I20260807 18:12:31.879541 22420 leader_election.cc:288] T 2c8fbebc008e4d1995d2065b893b01dc P bf17273192884a1e8a41b74403970a98 [CANDIDATE]: Term 1 election: Requested vote from peers f7ddbaeafa194da88d61994cf63503f6 (127.18.154.1:38089), f5c386f2052b4c86a1c451abcc9ca25e (127.18.154.3:43031)
I20260807 18:12:31.879987 21814 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c8fbebc008e4d1995d2065b893b01dc" candidate_uuid: "bf17273192884a1e8a41b74403970a98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7ddbaeafa194da88d61994cf63503f6"
I20260807 18:12:31.880131 22187 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c8fbebc008e4d1995d2065b893b01dc" candidate_uuid: "bf17273192884a1e8a41b74403970a98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5c386f2052b4c86a1c451abcc9ca25e"
I20260807 18:12:31.880203 21814 raft_consensus.cc:3037] T 2c8fbebc008e4d1995d2065b893b01dc P f7ddbaeafa194da88d61994cf63503f6 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:12:31.880375 22187 raft_consensus.cc:3037] T 2c8fbebc008e4d1995d2065b893b01dc P f5c386f2052b4c86a1c451abcc9ca25e [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:12:31.882712 21814 raft_consensus.cc:2454] T 2c8fbebc008e4d1995d2065b893b01dc P f7ddbaeafa194da88d61994cf63503f6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bf17273192884a1e8a41b74403970a98 in term 1.
I20260807 18:12:31.883206 21907 leader_election.cc:302] T 2c8fbebc008e4d1995d2065b893b01dc P bf17273192884a1e8a41b74403970a98 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bf17273192884a1e8a41b74403970a98, f7ddbaeafa194da88d61994cf63503f6; no voters:
I20260807 18:12:31.883517 22420 raft_consensus.cc:2781] T 2c8fbebc008e4d1995d2065b893b01dc P bf17273192884a1e8a41b74403970a98 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:12:31.883905 22187 raft_consensus.cc:2454] T 2c8fbebc008e4d1995d2065b893b01dc P f5c386f2052b4c86a1c451abcc9ca25e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bf17273192884a1e8a41b74403970a98 in term 1.
I20260807 18:12:31.884493 22420 raft_consensus.cc:686] T 2c8fbebc008e4d1995d2065b893b01dc P bf17273192884a1e8a41b74403970a98 [term 1 LEADER]: Becoming Leader. State: Replica: bf17273192884a1e8a41b74403970a98, State: Running, Role: LEADER
I20260807 18:12:31.884917 22420 consensus_queue.cc:237] T 2c8fbebc008e4d1995d2065b893b01dc P bf17273192884a1e8a41b74403970a98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7ddbaeafa194da88d61994cf63503f6" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 38089 } } peers { permanent_uuid: "bf17273192884a1e8a41b74403970a98" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 39793 } } peers { permanent_uuid: "f5c386f2052b4c86a1c451abcc9ca25e" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 43031 } }
I20260807 18:12:31.888722 21592 catalog_manager.cc:5495] T 2c8fbebc008e4d1995d2065b893b01dc P bf17273192884a1e8a41b74403970a98 reported cstate change: term changed from 0 to 1, leader changed from <none> to bf17273192884a1e8a41b74403970a98 (127.18.154.2). New cstate: current_term: 1 leader_uuid: "bf17273192884a1e8a41b74403970a98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7ddbaeafa194da88d61994cf63503f6" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 38089 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bf17273192884a1e8a41b74403970a98" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 39793 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5c386f2052b4c86a1c451abcc9ca25e" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 43031 } health_report { overall_health: UNKNOWN } } }
I20260807 18:12:31.938730 22441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:31.930812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36952 (local address 127.18.154.2:39793)
0807 18:12:31.932183 (+ 1371us) server_negotiation.cc:206] Beginning negotiation
0807 18:12:31.932188 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:12:31.932208 (+ 20us) server_negotiation.cc:407] Connection header received
0807 18:12:31.932300 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:12:31.932304 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:12:31.932385 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:12:31.932618 (+ 233us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:12:31.933221 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:31.934533 (+ 1312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:31.936082 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:31.936574 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:31.936724 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:31.937276 (+ 552us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:12:31.937988 (+ 712us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:12:31.938177 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0807 18:12:31.938253 (+ 76us) server_negotiation.cc:299] Negotiation successful
0807 18:12:31.938348 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1211,"thread_start_us":938,"threads_started":1}
I20260807 18:12:31.957675 22437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:31.951404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.2:36535 (local address 127.18.154.1:38089)
0807 18:12:31.951587 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0807 18:12:31.951592 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:12:31.952286 (+ 694us) server_negotiation.cc:407] Connection header received
0807 18:12:31.952469 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:12:31.952473 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:12:31.952541 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:12:31.952622 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:12:31.953108 (+ 486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:31.953939 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:31.955976 (+ 2037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:31.956447 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:31.956543 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:31.957014 (+ 471us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:12:31.957134 (+ 120us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:12:31.957202 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0807 18:12:31.957339 (+ 137us) server_negotiation.cc:299] Negotiation successful
0807 18:12:31.957406 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260807 18:12:31.958122 22447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:31.950871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.1:38089 (local address 127.18.154.2:36535)
0807 18:12:31.952192 (+ 1321us) negotiation.cc:107] Waiting for socket to connect
0807 18:12:31.952209 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 18:12:31.952334 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:12:31.952653 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:12:31.952656 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:12:31.952665 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 18:12:31.952976 (+ 311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:12:31.952984 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:12:31.954082 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:12:31.954087 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:12:31.955844 (+ 1757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:12:31.955858 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:12:31.956615 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:12:31.956620 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:12:31.956778 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:31.956845 (+ 67us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 18:12:31.957228 (+ 383us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 18:12:31.957237 (+ 9us) client_negotiation.cc:769] Sending connection context
0807 18:12:31.957300 (+ 63us) client_negotiation.cc:240] Negotiation successful
0807 18:12:31.957407 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1160,"thread_start_us":968,"threads_started":1}
I20260807 18:12:31.967557 22187 raft_consensus.cc:1261] T 2c8fbebc008e4d1995d2065b893b01dc P f5c386f2052b4c86a1c451abcc9ca25e [term 1 FOLLOWER]: Refusing update from remote peer bf17273192884a1e8a41b74403970a98: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:12:31.967924 21814 raft_consensus.cc:1261] T 2c8fbebc008e4d1995d2065b893b01dc P f7ddbaeafa194da88d61994cf63503f6 [term 1 FOLLOWER]: Refusing update from remote peer bf17273192884a1e8a41b74403970a98: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:12:31.968338 22420 consensus_queue.cc:1035] T 2c8fbebc008e4d1995d2065b893b01dc P bf17273192884a1e8a41b74403970a98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5c386f2052b4c86a1c451abcc9ca25e" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 43031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:12:31.969129 22420 consensus_queue.cc:1035] T 2c8fbebc008e4d1995d2065b893b01dc P bf17273192884a1e8a41b74403970a98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f7ddbaeafa194da88d61994cf63503f6" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 38089 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:12:32.002435 22441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:31.994640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.1:42225 (local address 127.18.154.2:39793)
0807 18:12:31.994789 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0807 18:12:31.994795 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:12:31.995822 (+ 1027us) server_negotiation.cc:407] Connection header received
0807 18:12:31.996037 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:12:31.996046 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:12:31.996112 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:12:31.996220 (+ 108us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:12:31.996846 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:31.997884 (+ 1038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:32.000519 (+ 2635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:32.001050 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:32.001226 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:32.001652 (+ 426us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:12:32.001759 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:12:32.001872 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0807 18:12:32.002013 (+ 141us) server_negotiation.cc:299] Negotiation successful
0807 18:12:32.002099 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260807 18:12:32.002761 22454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:31.994626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.2:39793 (local address 127.18.154.1:42225)
0807 18:12:31.995718 (+ 1092us) negotiation.cc:107] Waiting for socket to connect
0807 18:12:31.995734 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 18:12:31.995892 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:12:31.996242 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:12:31.996246 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:12:31.996258 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 18:12:31.996671 (+ 413us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:12:31.996683 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:12:31.998026 (+ 1343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:12:31.998032 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:12:32.000369 (+ 2337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:12:32.000383 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:12:32.001237 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:12:32.001243 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:12:32.001407 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:32.001449 (+ 42us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 18:12:32.001876 (+ 427us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 18:12:32.001891 (+ 15us) client_negotiation.cc:769] Sending connection context
0807 18:12:32.001991 (+ 100us) client_negotiation.cc:240] Negotiation successful
0807 18:12:32.002140 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":901,"thread_start_us":727,"threads_started":1}
W20260807 18:12:32.004318 22380 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260807 18:12:33.633734 21594 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58856:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:12:33.633994 21594 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58856:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:12:33.638526 21594 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6fff555ae1b94e1494e4fb470dad5674: Sending DeleteTablet for 3 replicas of tablet 2c8fbebc008e4d1995d2065b893b01dc
I20260807 18:12:33.639547 21784 tablet_service.cc:1514] Processing DeleteTablet for tablet 2c8fbebc008e4d1995d2065b893b01dc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:12:33 UTC) from {username='kudu', principal='kudu/127.18.154.62@KRBTEST.COM'} at 127.0.0.1:55412
I20260807 18:12:33.639596 21967 tablet_service.cc:1514] Processing DeleteTablet for tablet 2c8fbebc008e4d1995d2065b893b01dc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:12:33 UTC) from {username='kudu', principal='kudu/127.18.154.62@KRBTEST.COM'} at 127.0.0.1:36944
I20260807 18:12:33.639750 22160 tablet_service.cc:1514] Processing DeleteTablet for tablet 2c8fbebc008e4d1995d2065b893b01dc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:12:33 UTC) from {username='kudu', principal='kudu/127.18.154.62@KRBTEST.COM'} at 127.0.0.1:45078
I20260807 18:12:33.640413 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 21695
I20260807 18:12:33.652235 22487 tablet_replica.cc:330] T 2c8fbebc008e4d1995d2065b893b01dc P f7ddbaeafa194da88d61994cf63503f6: stopping tablet replica
I20260807 18:12:33.653290 22487 raft_consensus.cc:2229] T 2c8fbebc008e4d1995d2065b893b01dc P f7ddbaeafa194da88d61994cf63503f6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:12:33.654175 22487 raft_consensus.cc:2258] T 2c8fbebc008e4d1995d2065b893b01dc P f7ddbaeafa194da88d61994cf63503f6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:12:33.656324 22487 ts_tablet_manager.cc:1905] T 2c8fbebc008e4d1995d2065b893b01dc P f7ddbaeafa194da88d61994cf63503f6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:12:33.658119 22488 tablet_replica.cc:330] T 2c8fbebc008e4d1995d2065b893b01dc P bf17273192884a1e8a41b74403970a98: stopping tablet replica
I20260807 18:12:33.658562 22488 raft_consensus.cc:2229] T 2c8fbebc008e4d1995d2065b893b01dc P bf17273192884a1e8a41b74403970a98 [term 1 LEADER]: Raft consensus shutting down.
I20260807 18:12:33.659345 22488 raft_consensus.cc:2258] T 2c8fbebc008e4d1995d2065b893b01dc P bf17273192884a1e8a41b74403970a98 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:12:33.660430 22488 ts_tablet_manager.cc:1905] T 2c8fbebc008e4d1995d2065b893b01dc P bf17273192884a1e8a41b74403970a98: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:12:33.662128 22489 tablet_replica.cc:330] T 2c8fbebc008e4d1995d2065b893b01dc P f5c386f2052b4c86a1c451abcc9ca25e: stopping tablet replica
I20260807 18:12:33.663035 22489 raft_consensus.cc:2229] T 2c8fbebc008e4d1995d2065b893b01dc P f5c386f2052b4c86a1c451abcc9ca25e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:12:33.665516 22489 raft_consensus.cc:2258] T 2c8fbebc008e4d1995d2065b893b01dc P f5c386f2052b4c86a1c451abcc9ca25e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:12:33.667276 22487 ts_tablet_manager.cc:1918] T 2c8fbebc008e4d1995d2065b893b01dc P f7ddbaeafa194da88d61994cf63503f6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 18:12:33.667431 22488 ts_tablet_manager.cc:1918] T 2c8fbebc008e4d1995d2065b893b01dc P bf17273192884a1e8a41b74403970a98: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 18:12:33.667198 22489 ts_tablet_manager.cc:1905] T 2c8fbebc008e4d1995d2065b893b01dc P f5c386f2052b4c86a1c451abcc9ca25e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:12:33.667652 22488 log.cc:1196] T 2c8fbebc008e4d1995d2065b893b01dc P bf17273192884a1e8a41b74403970a98: Deleting WAL directory at /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126328808105-19048-0/minicluster-data/ts-1/wal/wals/2c8fbebc008e4d1995d2065b893b01dc
I20260807 18:12:33.668354 22488 ts_tablet_manager.cc:1939] T 2c8fbebc008e4d1995d2065b893b01dc P bf17273192884a1e8a41b74403970a98: Deleting consensus metadata
I20260807 18:12:33.668578 22487 log.cc:1196] T 2c8fbebc008e4d1995d2065b893b01dc P f7ddbaeafa194da88d61994cf63503f6: Deleting WAL directory at /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126328808105-19048-0/minicluster-data/ts-0/wal/wals/2c8fbebc008e4d1995d2065b893b01dc
I20260807 18:12:33.669162 22487 ts_tablet_manager.cc:1939] T 2c8fbebc008e4d1995d2065b893b01dc P f7ddbaeafa194da88d61994cf63503f6: Deleting consensus metadata
I20260807 18:12:33.669796 21572 catalog_manager.cc:4841] TS bf17273192884a1e8a41b74403970a98 (127.18.154.2:39793): tablet 2c8fbebc008e4d1995d2065b893b01dc (table test-table [id=fd81d65f77864250abd5c398d546fb29]) successfully deleted
I20260807 18:12:33.670336 21575 catalog_manager.cc:4841] TS f7ddbaeafa194da88d61994cf63503f6 (127.18.154.1:38089): tablet 2c8fbebc008e4d1995d2065b893b01dc (table test-table [id=fd81d65f77864250abd5c398d546fb29]) successfully deleted
Aug 07 18:12:33 dist-test-slave-gkw7 krb5kdc[21531](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126351, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.18.154.1@KRBTEST.COM
I20260807 18:12:33.679595 22489 ts_tablet_manager.cc:1918] T 2c8fbebc008e4d1995d2065b893b01dc P f5c386f2052b4c86a1c451abcc9ca25e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 18:12:33.679893 22489 log.cc:1196] T 2c8fbebc008e4d1995d2065b893b01dc P f5c386f2052b4c86a1c451abcc9ca25e: Deleting WAL directory at /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786126328808105-19048-0/minicluster-data/ts-2/wal/wals/2c8fbebc008e4d1995d2065b893b01dc
I20260807 18:12:33.680387 22489 ts_tablet_manager.cc:1939] T 2c8fbebc008e4d1995d2065b893b01dc P f5c386f2052b4c86a1c451abcc9ca25e: Deleting consensus metadata
I20260807 18:12:33.681515 21571 catalog_manager.cc:4841] TS f5c386f2052b4c86a1c451abcc9ca25e (127.18.154.3:43031): tablet 2c8fbebc008e4d1995d2065b893b01dc (table test-table [id=fd81d65f77864250abd5c398d546fb29]) successfully deleted
I20260807 18:12:33.685346 22492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:33.643415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55450 (local address 127.18.154.1:38089)
0807 18:12:33.657921 (+ 14506us) server_negotiation.cc:206] Beginning negotiation
0807 18:12:33.657929 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:12:33.657959 (+ 30us) server_negotiation.cc:407] Connection header received
0807 18:12:33.658041 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:12:33.658047 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:12:33.658142 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:12:33.660411 (+ 2269us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:12:33.660640 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:33.662643 (+ 2003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:33.665247 (+ 2604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:33.668122 (+ 2875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:33.668305 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:33.678621 (+ 10316us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:12:33.678660 (+ 39us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:12:33.678678 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:12:33.678725 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:12:33.682317 (+ 3592us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:12:33.683076 (+ 759us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:12:33.683089 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:12:33.683092 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:12:33.683245 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:12:33.683668 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:12:33.683678 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:12:33.683680 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:12:33.684075 (+ 395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:12:33.684404 (+ 329us) server_negotiation.cc:1035] Waiting for connection context
0807 18:12:33.684746 (+ 342us) server_negotiation.cc:299] Negotiation successful
0807 18:12:33.684981 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14339,"thread_start_us":14838,"threads_started":1}
W20260807 18:12:33.686102 21871 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55450
W20260807 18:12:33.686923 19048 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 21695: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 18:12:33.687160 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 21695
I20260807 18:12:33.724648 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 21881
Aug 07 18:12:33 dist-test-slave-gkw7 krb5kdc[21531](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126351, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.18.154.2@KRBTEST.COM
I20260807 18:12:33.739924 22493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:33.725492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36968 (local address 127.18.154.2:39793)
0807 18:12:33.726010 (+ 518us) server_negotiation.cc:206] Beginning negotiation
0807 18:12:33.726015 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:12:33.726031 (+ 16us) server_negotiation.cc:407] Connection header received
0807 18:12:33.726090 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:12:33.726094 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:12:33.726158 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:12:33.726249 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:12:33.727467 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:33.728305 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:33.729751 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:33.730191 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:33.730287 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:33.734554 (+ 4267us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:12:33.734581 (+ 27us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:12:33.734598 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:12:33.734633 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:12:33.737363 (+ 2730us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:12:33.738038 (+ 675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:12:33.738045 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:12:33.738048 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:12:33.738187 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:12:33.738497 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:12:33.738501 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:12:33.738503 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:12:33.738878 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:12:33.739089 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0807 18:12:33.739353 (+ 264us) server_negotiation.cc:299] Negotiation successful
0807 18:12:33.739530 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":405,"thread_start_us":276,"threads_started":1}
W20260807 18:12:33.740660 22064 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36968
W20260807 18:12:33.741302 19048 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 21881: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 18:12:33.741494 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 21881
I20260807 18:12:33.772487 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 22076
Aug 07 18:12:33 dist-test-slave-gkw7 krb5kdc[21531](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126351, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.18.154.3@KRBTEST.COM
I20260807 18:12:33.788116 22494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:33.773282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45110 (local address 127.18.154.3:43031)
0807 18:12:33.773894 (+ 612us) server_negotiation.cc:206] Beginning negotiation
0807 18:12:33.773899 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:12:33.773915 (+ 16us) server_negotiation.cc:407] Connection header received
0807 18:12:33.773971 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:12:33.773974 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:12:33.774033 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:12:33.774129 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:12:33.775487 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:33.776321 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:33.777722 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:33.778109 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:33.778208 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:33.782487 (+ 4279us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:12:33.782504 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:12:33.782522 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:12:33.782569 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:12:33.785654 (+ 3085us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:12:33.786337 (+ 683us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:12:33.786345 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:12:33.786347 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:12:33.786476 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:12:33.786802 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:12:33.786805 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:12:33.786807 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:12:33.787215 (+ 408us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:12:33.787428 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0807 18:12:33.787689 (+ 261us) server_negotiation.cc:299] Negotiation successful
0807 18:12:33.787855 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":510,"thread_start_us":372,"threads_started":1}
W20260807 18:12:33.788902 22249 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45110
W20260807 18:12:33.789599 19048 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 22076: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 18:12:33.789796 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 22076
I20260807 18:12:33.822865 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 21549
I20260807 18:12:33.824235 21650 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:12:33.947075 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 21549
2026-08-07T18:12:33Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (9343 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126328808105-19048-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:12:34 dist-test-slave-gkw7 krb5kdc[22499](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:12:34 dist-test-slave-gkw7 krb5kdc[22499](info): set up 2 sockets
Aug 07 18:12:34 dist-test-slave-gkw7 krb5kdc[22499](info): commencing operation
krb5kdc: starting...
W20260807 18:12:35.236547 19048 mini_kdc.cc:121] Time spent starting KDC: real 1.232s user 0.003s sys 0.019s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126328808105-19048-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126328808105-19048-0/krb5kdc/test-user.keytab.
Aug 07 18:12:35 dist-test-slave-gkw7 krb5kdc[22499](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126355, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:12:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:12:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.154.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.154.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.154.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.62.keytab.
Entry for principal kudu/127.18.154.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.62.keytab.
I20260807 18:12:35.467579 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126328808105-19048-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126328808105-19048-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126328808105-19048-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.154.62:35307
--webserver_interface=127.18.154.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.62.keytab
--principal=kudu/127.18.154.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.154.20:41019
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.154.62:35307
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126328808105-19048-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126328808105-19048-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126328808105-19048-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:12:35.812214 22514 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:12:35.812608 22514 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:12:35.812794 22514 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:12:35.820863 22514 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:12:35.821055 22514 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:12:35.821163 22514 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:12:35.821257 22514 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:12:35.821354 22514 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:12:35.831928 22514 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.154.20:41019
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126328808105-19048-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126328808105-19048-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.154.62:35307
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.62.keytab
--principal=kudu/127.18.154.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.154.62:35307
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.154.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126328808105-19048-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:12:35.833464 22514 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:12:35.834911 22514 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:12:35.848117 22520 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:12:35.848145 22521 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:12:35.848266 22525 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:12:35.848657 22514 server_base.cc:1034] running on GCE node
I20260807 18:12:35.849614 22514 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:12:35.851125 22514 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:12:35.852375 22514 hybrid_clock.cc:648] HybridClock initialized: now 1786126355852329 us; error 33 us; skew 500 ppm
I20260807 18:12:35.852793 22514 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:12:35 dist-test-slave-gkw7 krb5kdc[22499](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126355, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:12:35.856521 22514 init.cc:377] Logged in from keytab as kudu/127.18.154.62@KRBTEST.COM (short username kudu)
I20260807 18:12:35.858140 22514 webserver.cc:460] Webserver started at http://127.18.154.62:34139/ using document root <none> and password file <none>
I20260807 18:12:35.858934 22514 fs_manager.cc:359] Metadata directory not provided
I20260807 18:12:35.859090 22514 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:12:35.859416 22514 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:12:35.862527 22514 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126328808105-19048-0/minicluster-data/master-0/data/instance:
uuid: "6dea32e988554908b6ee134c1d27b4e5"
format_stamp: "Formatted at 2026-08-07 18:12:35 on dist-test-slave-gkw7"
server_key: "a943917d01c54385c51583d0dfa15cd6"
server_key_iv: "6d848d93fee17c6ff99bcc50c0c44d1f"
server_key_version: "encryptionkey@0"
I20260807 18:12:35.863349 22514 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126328808105-19048-0/minicluster-data/master-0/wal/instance:
uuid: "6dea32e988554908b6ee134c1d27b4e5"
format_stamp: "Formatted at 2026-08-07 18:12:35 on dist-test-slave-gkw7"
server_key: "a943917d01c54385c51583d0dfa15cd6"
server_key_iv: "6d848d93fee17c6ff99bcc50c0c44d1f"
server_key_version: "encryptionkey@0"
I20260807 18:12:35.868695 22514 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 18:12:35.872289 22534 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:12:35.873730 22514 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:12:35.873919 22514 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126328808105-19048-0/minicluster-data/master-0/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126328808105-19048-0/minicluster-data/master-0/wal
uuid: "6dea32e988554908b6ee134c1d27b4e5"
format_stamp: "Formatted at 2026-08-07 18:12:35 on dist-test-slave-gkw7"
server_key: "a943917d01c54385c51583d0dfa15cd6"
server_key_iv: "6d848d93fee17c6ff99bcc50c0c44d1f"
server_key_version: "encryptionkey@0"
I20260807 18:12:35.874162 22514 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126328808105-19048-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126328808105-19048-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126328808105-19048-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:12:35.904919 22514 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:12:35.909626 22514 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:12:35.909934 22514 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:12:35.928840 22514 rpc_server.cc:307] RPC server started. Bound to: 127.18.154.62:35307
I20260807 18:12:35.928859 22610 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.154.62:35307 every 8 connection(s)
I20260807 18:12:35.930444 22514 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
I20260807 18:12:35.934446 22612 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:12:35.940272 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 22514
I20260807 18:12:35.940491 19048 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126328808105-19048-0/minicluster-data/master-0/wal/instance
I20260807 18:12:35.941293 19048 external_mini_cluster.cc:1428] Setting key 8369bb572bef69afef3fa9faf58b76fc
I20260807 18:12:35.946918 22612 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6dea32e988554908b6ee134c1d27b4e5: Bootstrap starting.
I20260807 18:12:35.950119 22612 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6dea32e988554908b6ee134c1d27b4e5: Neither blocks nor log segments found. Creating new log.
I20260807 18:12:35.951473 22612 log.cc:824] T 00000000000000000000000000000000 P 6dea32e988554908b6ee134c1d27b4e5: Log is configured to *not* fsync() on all Append() calls
I20260807 18:12:35.955024 22612 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6dea32e988554908b6ee134c1d27b4e5: No bootstrap required, opened a new log
Aug 07 18:12:35 dist-test-slave-gkw7 krb5kdc[22499](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126355, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.154.62@KRBTEST.COM
I20260807 18:12:35.963263 22612 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6dea32e988554908b6ee134c1d27b4e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6dea32e988554908b6ee134c1d27b4e5" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 35307 } }
I20260807 18:12:35.963690 22612 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6dea32e988554908b6ee134c1d27b4e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:12:35.963806 22612 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6dea32e988554908b6ee134c1d27b4e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6dea32e988554908b6ee134c1d27b4e5, State: Initialized, Role: FOLLOWER
I20260807 18:12:35.964496 22612 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6dea32e988554908b6ee134c1d27b4e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6dea32e988554908b6ee134c1d27b4e5" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 35307 } }
I20260807 18:12:35.964752 22612 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6dea32e988554908b6ee134c1d27b4e5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:12:35.964906 22612 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6dea32e988554908b6ee134c1d27b4e5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:12:35.965098 22612 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6dea32e988554908b6ee134c1d27b4e5 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:12:35.967113 22615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:35.943602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38546 (local address 127.18.154.62:35307)
0807 18:12:35.944507 (+ 905us) server_negotiation.cc:206] Beginning negotiation
0807 18:12:35.944520 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 18:12:35.944565 (+ 45us) server_negotiation.cc:407] Connection header received
0807 18:12:35.946382 (+ 1817us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:12:35.946406 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:12:35.946797 (+ 391us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:12:35.947192 (+ 395us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:12:35.948708 (+ 1516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:35.950273 (+ 1565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:35.952149 (+ 1876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:35.953187 (+ 1038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:35.953323 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:35.958517 (+ 5194us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:12:35.958535 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:12:35.958551 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:12:35.958600 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:12:35.962398 (+ 3798us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:12:35.963218 (+ 820us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:12:35.963228 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:12:35.963238 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:12:35.963441 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:12:35.963930 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:12:35.963939 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:12:35.963943 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:12:35.964571 (+ 628us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:12:35.964897 (+ 326us) server_negotiation.cc:1035] Waiting for connection context
0807 18:12:35.965419 (+ 522us) server_negotiation.cc:299] Negotiation successful
0807 18:12:35.965943 (+ 524us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":672,"thread_start_us":385,"threads_started":1}
I20260807 18:12:35.967978 22612 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6dea32e988554908b6ee134c1d27b4e5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6dea32e988554908b6ee134c1d27b4e5" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 35307 } }
I20260807 18:12:35.968595 22612 leader_election.cc:302] T 00000000000000000000000000000000 P 6dea32e988554908b6ee134c1d27b4e5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6dea32e988554908b6ee134c1d27b4e5; no voters:
I20260807 18:12:35.969331 22612 leader_election.cc:288] T 00000000000000000000000000000000 P 6dea32e988554908b6ee134c1d27b4e5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:12:35.969444 22619 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6dea32e988554908b6ee134c1d27b4e5 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:12:35.971169 22619 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6dea32e988554908b6ee134c1d27b4e5 [term 1 LEADER]: Becoming Leader. State: Replica: 6dea32e988554908b6ee134c1d27b4e5, State: Running, Role: LEADER
I20260807 18:12:35.971728 22619 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6dea32e988554908b6ee134c1d27b4e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6dea32e988554908b6ee134c1d27b4e5" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 35307 } }
I20260807 18:12:35.971912 22612 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6dea32e988554908b6ee134c1d27b4e5 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:12:35.975315 22621 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6dea32e988554908b6ee134c1d27b4e5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6dea32e988554908b6ee134c1d27b4e5. Latest consensus state: current_term: 1 leader_uuid: "6dea32e988554908b6ee134c1d27b4e5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6dea32e988554908b6ee134c1d27b4e5" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 35307 } } }
I20260807 18:12:35.975721 22621 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6dea32e988554908b6ee134c1d27b4e5 [sys.catalog]: This master's current role is: LEADER
I20260807 18:12:35.975449 22620 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6dea32e988554908b6ee134c1d27b4e5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6dea32e988554908b6ee134c1d27b4e5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6dea32e988554908b6ee134c1d27b4e5" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 35307 } } }
I20260807 18:12:35.975916 22620 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6dea32e988554908b6ee134c1d27b4e5 [sys.catalog]: This master's current role is: LEADER
I20260807 18:12:35.976671 22629 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:12:35.988816 22629 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:12:35.998864 22629 catalog_manager.cc:1347] Generated new cluster ID: 0691f90740594c80931ea4713b0e9fc9
I20260807 18:12:35.998996 22629 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:12:36.017221 22629 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:12:36.018630 22629 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:12:36.027691 22629 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6dea32e988554908b6ee134c1d27b4e5: Generated new TSK 0
I20260807 18:12:36.028573 22629 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.154.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.154.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.154.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.1.keytab.
Entry for principal kudu/127.18.154.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.1.keytab.
I20260807 18:12:36.081707 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126328808105-19048-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126328808105-19048-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126328808105-19048-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126328808105-19048-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.154.1:0
--local_ip_for_outbound_sockets=127.18.154.1
--webserver_interface=127.18.154.1
--webserver_port=0
--tserver_master_addrs=127.18.154.62:35307
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.1.keytab
--principal=kudu/127.18.154.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.154.20:41019
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126328808105-19048-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126328808105-19048-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126328808105-19048-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:12:36.427534 22653 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:12:36.427910 22653 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:12:36.428067 22653 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:12:36.435568 22653 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:12:36.435707 22653 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:12:36.435905 22653 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.154.1
I20260807 18:12:36.446410 22653 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.154.20:41019
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126328808105-19048-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126328808105-19048-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.1.keytab
--principal=kudu/127.18.154.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.154.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126328808105-19048-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.154.1
--webserver_port=0
--tserver_master_addrs=127.18.154.62:35307
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.154.1
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126328808105-19048-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:12:36.447793 22653 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:12:36.449154 22653 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:12:36.463469 22665 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:12:36.463433 22663 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:12:36.463505 22662 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:12:36.464435 22653 server_base.cc:1034] running on GCE node
I20260807 18:12:36.465109 22653 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:12:36.466274 22653 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:12:36.467505 22653 hybrid_clock.cc:648] HybridClock initialized: now 1786126356467439 us; error 67 us; skew 500 ppm
I20260807 18:12:36.467931 22653 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:12:36 dist-test-slave-gkw7 krb5kdc[22499](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126356, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:12:36.471272 22653 init.cc:377] Logged in from keytab as kudu/127.18.154.1@KRBTEST.COM (short username kudu)
I20260807 18:12:36.473090 22653 webserver.cc:460] Webserver started at http://127.18.154.1:44911/ using document root <none> and password file <none>
I20260807 18:12:36.473898 22653 fs_manager.cc:359] Metadata directory not provided
I20260807 18:12:36.474033 22653 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:12:36.474368 22653 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:12:36.477392 22653 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126328808105-19048-0/minicluster-data/ts-0/data/instance:
uuid: "d57d9f890584465fbfc0bcae5567aa89"
format_stamp: "Formatted at 2026-08-07 18:12:36 on dist-test-slave-gkw7"
server_key: "71d5ca35bf46040eaf262e177ec86dfe"
server_key_iv: "e85dd381706bda35b1f1cf036467ceae"
server_key_version: "encryptionkey@0"
I20260807 18:12:36.478353 22653 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126328808105-19048-0/minicluster-data/ts-0/wal/instance:
uuid: "d57d9f890584465fbfc0bcae5567aa89"
format_stamp: "Formatted at 2026-08-07 18:12:36 on dist-test-slave-gkw7"
server_key: "71d5ca35bf46040eaf262e177ec86dfe"
server_key_iv: "e85dd381706bda35b1f1cf036467ceae"
server_key_version: "encryptionkey@0"
I20260807 18:12:36.484042 22653 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260807 18:12:36.487861 22672 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:12:36.489320 22653 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 18:12:36.489501 22653 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126328808105-19048-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126328808105-19048-0/minicluster-data/ts-0/wal
uuid: "d57d9f890584465fbfc0bcae5567aa89"
format_stamp: "Formatted at 2026-08-07 18:12:36 on dist-test-slave-gkw7"
server_key: "71d5ca35bf46040eaf262e177ec86dfe"
server_key_iv: "e85dd381706bda35b1f1cf036467ceae"
server_key_version: "encryptionkey@0"
I20260807 18:12:36.489794 22653 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126328808105-19048-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126328808105-19048-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126328808105-19048-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:12:36.516041 22653 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:12:36.520903 22653 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:12:36.521229 22653 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:12:36.522359 22653 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:12:36.523877 22653 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:12:36.523957 22653 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:12:36.524084 22653 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:12:36.524142 22653 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:12:36.559491 22653 rpc_server.cc:307] RPC server started. Bound to: 127.18.154.1:36673
I20260807 18:12:36.559543 22821 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.154.1:36673 every 8 connection(s)
I20260807 18:12:36.560897 22653 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126328808105-19048-0/minicluster-data/ts-0/data/info.pb
I20260807 18:12:36.566066 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 22653
I20260807 18:12:36.566290 19048 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126328808105-19048-0/minicluster-data/ts-0/wal/instance
I20260807 18:12:36.567124 19048 external_mini_cluster.cc:1428] Setting key 5bffe01f956c2e24850c043d54e247d4
Aug 07 18:12:36 dist-test-slave-gkw7 krb5kdc[22499](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126356, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.1@KRBTEST.COM for kudu/127.18.154.62@KRBTEST.COM
I20260807 18:12:36.584764 22826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:36.563703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.1:58661 (local address 127.18.154.62:35307)
0807 18:12:36.564180 (+ 477us) server_negotiation.cc:206] Beginning negotiation
0807 18:12:36.564185 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:12:36.565104 (+ 919us) server_negotiation.cc:407] Connection header received
0807 18:12:36.567890 (+ 2786us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:12:36.567896 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:12:36.567990 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:12:36.568119 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:12:36.569963 (+ 1844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:36.570689 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:36.573303 (+ 2614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:36.573911 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:36.574044 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:36.578559 (+ 4515us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:12:36.578564 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:12:36.578568 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:12:36.578603 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:12:36.581315 (+ 2712us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:12:36.582184 (+ 869us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:12:36.582189 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:12:36.582192 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:12:36.582328 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:12:36.582817 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:12:36.582821 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:12:36.582823 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:12:36.583099 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:12:36.583275 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0807 18:12:36.584094 (+ 819us) server_negotiation.cc:299] Negotiation successful
0807 18:12:36.584373 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":371,"thread_start_us":270,"threads_started":1}
I20260807 18:12:36.585878 22827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:36.564229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:35307 (local address 127.18.154.1:58661)
0807 18:12:36.564941 (+ 712us) negotiation.cc:107] Waiting for socket to connect
0807 18:12:36.564979 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0807 18:12:36.567534 (+ 2555us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:12:36.568380 (+ 846us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:12:36.568389 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:12:36.568917 (+ 528us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:12:36.569764 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:12:36.569795 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:12:36.570874 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:12:36.570883 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:12:36.573115 (+ 2232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:12:36.573134 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:12:36.574162 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:12:36.574171 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:12:36.574538 (+ 367us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:36.575356 (+ 818us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:12:36.575393 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:12:36.578297 (+ 2904us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:12:36.581484 (+ 3187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:12:36.581494 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:12:36.581513 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:12:36.582039 (+ 526us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:12:36.582460 (+ 421us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:12:36.582465 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:12:36.582469 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:12:36.582671 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:12:36.583247 (+ 576us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:12:36.583256 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:12:36.583673 (+ 417us) client_negotiation.cc:769] Sending connection context
0807 18:12:36.583972 (+ 299us) client_negotiation.cc:240] Negotiation successful
0807 18:12:36.584534 (+ 562us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":485,"thread_start_us":273,"threads_started":1}
I20260807 18:12:36.587670 22822 heartbeater.cc:347] Connected to a master server at 127.18.154.62:35307
I20260807 18:12:36.588058 22822 heartbeater.cc:464] Registering TS with master...
I20260807 18:12:36.589066 22822 heartbeater.cc:511] Master 127.18.154.62:35307 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.18.154.2@KRBTEST.COM; defaulting to no policy
I20260807 18:12:36.591594 22557 ts_manager.cc:194] Registered new tserver with Master: d57d9f890584465fbfc0bcae5567aa89 (127.18.154.1:36673)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.154.2@KRBTEST.COM" created.
I20260807 18:12:36.593431 22557 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.154.1@KRBTEST.COM'} at 127.18.154.1:58661
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.154.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.2.keytab.
Entry for principal kudu/127.18.154.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.2.keytab.
I20260807 18:12:36.621011 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126328808105-19048-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126328808105-19048-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126328808105-19048-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126328808105-19048-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.154.2:0
--local_ip_for_outbound_sockets=127.18.154.2
--webserver_interface=127.18.154.2
--webserver_port=0
--tserver_master_addrs=127.18.154.62:35307
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.2.keytab
--principal=kudu/127.18.154.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.154.20:41019
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126328808105-19048-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126328808105-19048-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126328808105-19048-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:12:36.966773 22831 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:12:36.967097 22831 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:12:36.967263 22831 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:12:36.974870 22831 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:12:36.975000 22831 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:12:36.975178 22831 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.154.2
I20260807 18:12:36.985478 22831 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.154.20:41019
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126328808105-19048-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126328808105-19048-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.2.keytab
--principal=kudu/127.18.154.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.154.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126328808105-19048-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.154.2
--webserver_port=0
--tserver_master_addrs=127.18.154.62:35307
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.154.2
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126328808105-19048-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:12:36.987342 22831 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:12:36.989112 22831 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:12:37.003082 22842 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:12:37.003086 22840 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:12:37.003324 22845 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:12:37.004531 22831 server_base.cc:1034] running on GCE node
I20260807 18:12:37.005296 22831 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:12:37.006510 22831 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:12:37.007738 22831 hybrid_clock.cc:648] HybridClock initialized: now 1786126357007663 us; error 76 us; skew 500 ppm
I20260807 18:12:37.008113 22831 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:12:37 dist-test-slave-gkw7 krb5kdc[22499](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126357, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:12:37.011957 22831 init.cc:377] Logged in from keytab as kudu/127.18.154.2@KRBTEST.COM (short username kudu)
I20260807 18:12:37.013818 22831 webserver.cc:460] Webserver started at http://127.18.154.2:36543/ using document root <none> and password file <none>
I20260807 18:12:37.014596 22831 fs_manager.cc:359] Metadata directory not provided
I20260807 18:12:37.014714 22831 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:12:37.015036 22831 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:12:37.018529 22831 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126328808105-19048-0/minicluster-data/ts-1/data/instance:
uuid: "03409e41bde44d4e8616fb3b58b96aa7"
format_stamp: "Formatted at 2026-08-07 18:12:37 on dist-test-slave-gkw7"
server_key: "253a7f106cfc45e1982ad253a5f4989e"
server_key_iv: "9fc591a40ce8eaf17fd8c2f4e7398484"
server_key_version: "encryptionkey@0"
I20260807 18:12:37.019379 22831 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126328808105-19048-0/minicluster-data/ts-1/wal/instance:
uuid: "03409e41bde44d4e8616fb3b58b96aa7"
format_stamp: "Formatted at 2026-08-07 18:12:37 on dist-test-slave-gkw7"
server_key: "253a7f106cfc45e1982ad253a5f4989e"
server_key_iv: "9fc591a40ce8eaf17fd8c2f4e7398484"
server_key_version: "encryptionkey@0"
I20260807 18:12:37.025350 22831 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.002s
I20260807 18:12:37.029296 22853 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:12:37.030916 22831 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:12:37.031073 22831 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126328808105-19048-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126328808105-19048-0/minicluster-data/ts-1/wal
uuid: "03409e41bde44d4e8616fb3b58b96aa7"
format_stamp: "Formatted at 2026-08-07 18:12:37 on dist-test-slave-gkw7"
server_key: "253a7f106cfc45e1982ad253a5f4989e"
server_key_iv: "9fc591a40ce8eaf17fd8c2f4e7398484"
server_key_version: "encryptionkey@0"
I20260807 18:12:37.031286 22831 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126328808105-19048-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126328808105-19048-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126328808105-19048-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:12:37.052069 22831 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:12:37.057253 22831 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:12:37.057658 22831 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:12:37.058818 22831 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:12:37.060457 22831 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:12:37.060539 22831 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:12:37.060711 22831 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:12:37.060767 22831 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:12:37.099114 22831 rpc_server.cc:307] RPC server started. Bound to: 127.18.154.2:37925
I20260807 18:12:37.099152 23017 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.154.2:37925 every 8 connection(s)
I20260807 18:12:37.100625 22831 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126328808105-19048-0/minicluster-data/ts-1/data/info.pb
I20260807 18:12:37.104898 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 22831
I20260807 18:12:37.105104 19048 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126328808105-19048-0/minicluster-data/ts-1/wal/instance
I20260807 18:12:37.105738 19048 external_mini_cluster.cc:1428] Setting key 0f10553a46d66fcbb200f8798fdeb2b4
Aug 07 18:12:37 dist-test-slave-gkw7 krb5kdc[22499](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126357, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.2@KRBTEST.COM for kudu/127.18.154.62@KRBTEST.COM
I20260807 18:12:37.126753 23022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:37.103508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.2:34545 (local address 127.18.154.62:35307)
0807 18:12:37.104014 (+ 506us) server_negotiation.cc:206] Beginning negotiation
0807 18:12:37.104022 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:12:37.105394 (+ 1372us) server_negotiation.cc:407] Connection header received
0807 18:12:37.107762 (+ 2368us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:12:37.107770 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:12:37.107859 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:12:37.107999 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:12:37.110399 (+ 2400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:37.111150 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:37.113835 (+ 2685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:37.114319 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:37.114436 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:37.118896 (+ 4460us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:12:37.118906 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:12:37.118914 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:12:37.118966 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:12:37.123001 (+ 4035us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:12:37.123889 (+ 888us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:12:37.123897 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:12:37.123903 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:12:37.124095 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:12:37.124569 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:12:37.124577 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:12:37.124583 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:12:37.124981 (+ 398us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:12:37.125156 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0807 18:12:37.125966 (+ 810us) server_negotiation.cc:299] Negotiation successful
0807 18:12:37.126256 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":366,"thread_start_us":246,"threads_started":1}
I20260807 18:12:37.127523 23023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:37.104233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:35307 (local address 127.18.154.2:34545)
0807 18:12:37.105143 (+ 910us) negotiation.cc:107] Waiting for socket to connect
0807 18:12:37.105198 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0807 18:12:37.107432 (+ 2234us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:12:37.108325 (+ 893us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:12:37.108338 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:12:37.109183 (+ 845us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:12:37.110151 (+ 968us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:12:37.110182 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:12:37.111345 (+ 1163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:12:37.111353 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:12:37.113663 (+ 2310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:12:37.113683 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:12:37.114582 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:12:37.114592 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:12:37.114934 (+ 342us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:37.115774 (+ 840us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:12:37.115820 (+ 46us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:12:37.118572 (+ 2752us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:12:37.123205 (+ 4633us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:12:37.123214 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:12:37.123251 (+ 37us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:12:37.123728 (+ 477us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:12:37.124246 (+ 518us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:12:37.124249 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:12:37.124252 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:12:37.124439 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:12:37.125105 (+ 666us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:12:37.125111 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:12:37.125472 (+ 361us) client_negotiation.cc:769] Sending connection context
0807 18:12:37.125817 (+ 345us) client_negotiation.cc:240] Negotiation successful
0807 18:12:37.126258 (+ 441us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":577,"thread_start_us":326,"threads_started":1}
I20260807 18:12:37.129354 23018 heartbeater.cc:347] Connected to a master server at 127.18.154.62:35307
I20260807 18:12:37.129726 23018 heartbeater.cc:464] Registering TS with master...
I20260807 18:12:37.130496 23018 heartbeater.cc:511] Master 127.18.154.62:35307 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.18.154.3@KRBTEST.COM; defaulting to no policy
I20260807 18:12:37.132066 22557 ts_manager.cc:194] Registered new tserver with Master: 03409e41bde44d4e8616fb3b58b96aa7 (127.18.154.2:37925)
I20260807 18:12:37.132866 22557 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.154.2@KRBTEST.COM'} at 127.18.154.2:34545
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.154.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.154.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.3.keytab.
Entry for principal kudu/127.18.154.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.3.keytab.
I20260807 18:12:37.161870 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126328808105-19048-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126328808105-19048-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126328808105-19048-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126328808105-19048-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.154.3:0
--local_ip_for_outbound_sockets=127.18.154.3
--webserver_interface=127.18.154.3
--webserver_port=0
--tserver_master_addrs=127.18.154.62:35307
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.3.keytab
--principal=kudu/127.18.154.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.154.20:41019
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126328808105-19048-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126328808105-19048-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126328808105-19048-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:12:37.514529 23029 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:12:37.514839 23029 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:12:37.514935 23029 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:12:37.522464 23029 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:12:37.522593 23029 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:12:37.522709 23029 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.154.3
I20260807 18:12:37.532678 23029 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.154.20:41019
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126328808105-19048-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126328808105-19048-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.3.keytab
--principal=kudu/127.18.154.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.154.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126328808105-19048-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.154.3
--webserver_port=0
--tserver_master_addrs=127.18.154.62:35307
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.154.3
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126328808105-19048-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:12:37.534103 23029 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:12:37.535512 23029 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:12:37.549835 23037 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:12:37.549921 23035 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:12:37.549898 23034 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:12:37.550621 23029 server_base.cc:1034] running on GCE node
I20260807 18:12:37.551307 23029 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:12:37.552347 23029 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:12:37.553664 23029 hybrid_clock.cc:648] HybridClock initialized: now 1786126357553574 us; error 75 us; skew 500 ppm
I20260807 18:12:37.554039 23029 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:12:37 dist-test-slave-gkw7 krb5kdc[22499](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126357, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:12:37.557718 23029 init.cc:377] Logged in from keytab as kudu/127.18.154.3@KRBTEST.COM (short username kudu)
I20260807 18:12:37.559590 23029 webserver.cc:460] Webserver started at http://127.18.154.3:38099/ using document root <none> and password file <none>
I20260807 18:12:37.560356 23029 fs_manager.cc:359] Metadata directory not provided
I20260807 18:12:37.560511 23029 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:12:37.560856 23029 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:12:37.564088 23029 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126328808105-19048-0/minicluster-data/ts-2/data/instance:
uuid: "af5ee4fe9a97406e96b54c59648bc526"
format_stamp: "Formatted at 2026-08-07 18:12:37 on dist-test-slave-gkw7"
server_key: "4a3a582c3fe9d7479303c18cb8e5329c"
server_key_iv: "506909d5f056db9eb10d4e8e543fa959"
server_key_version: "encryptionkey@0"
I20260807 18:12:37.564944 23029 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126328808105-19048-0/minicluster-data/ts-2/wal/instance:
uuid: "af5ee4fe9a97406e96b54c59648bc526"
format_stamp: "Formatted at 2026-08-07 18:12:37 on dist-test-slave-gkw7"
server_key: "4a3a582c3fe9d7479303c18cb8e5329c"
server_key_iv: "506909d5f056db9eb10d4e8e543fa959"
server_key_version: "encryptionkey@0"
I20260807 18:12:37.570773 23029 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260807 18:12:37.574582 23048 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:12:37.575958 23029 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 18:12:37.576153 23029 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126328808105-19048-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126328808105-19048-0/minicluster-data/ts-2/wal
uuid: "af5ee4fe9a97406e96b54c59648bc526"
format_stamp: "Formatted at 2026-08-07 18:12:37 on dist-test-slave-gkw7"
server_key: "4a3a582c3fe9d7479303c18cb8e5329c"
server_key_iv: "506909d5f056db9eb10d4e8e543fa959"
server_key_version: "encryptionkey@0"
I20260807 18:12:37.576385 23029 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126328808105-19048-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126328808105-19048-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126328808105-19048-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:12:37.592078 23029 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:12:37.596657 22822 heartbeater.cc:503] Master 127.18.154.62:35307 was elected leader, sending a full tablet report...
I20260807 18:12:37.596690 23029 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:12:37.597015 23029 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:12:37.598265 23029 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:12:37.599684 23029 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:12:37.599762 23029 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:12:37.599936 23029 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:12:37.599992 23029 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:12:37.633665 23029 rpc_server.cc:307] RPC server started. Bound to: 127.18.154.3:40383
I20260807 18:12:37.633684 23207 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.154.3:40383 every 8 connection(s)
I20260807 18:12:37.635195 23029 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126328808105-19048-0/minicluster-data/ts-2/data/info.pb
I20260807 18:12:37.635583 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 23029
I20260807 18:12:37.635776 19048 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786126328808105-19048-0/minicluster-data/ts-2/wal/instance
I20260807 18:12:37.636515 19048 external_mini_cluster.cc:1428] Setting key 6010720615c3fd6db929eba692cf18b6
Aug 07 18:12:37 dist-test-slave-gkw7 krb5kdc[22499](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126357, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.3@KRBTEST.COM for kudu/127.18.154.62@KRBTEST.COM
I20260807 18:12:37.656821 23214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:37.638444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.3:44039 (local address 127.18.154.62:35307)
0807 18:12:37.638947 (+ 503us) server_negotiation.cc:206] Beginning negotiation
0807 18:12:37.638954 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:12:37.640079 (+ 1125us) server_negotiation.cc:407] Connection header received
0807 18:12:37.642036 (+ 1957us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:12:37.642041 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:12:37.642145 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:12:37.642274 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:12:37.643845 (+ 1571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:37.644621 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:37.646479 (+ 1858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:37.646869 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:37.647004 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:37.650835 (+ 3831us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:12:37.650844 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:12:37.650849 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:12:37.650897 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:12:37.654015 (+ 3118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:12:37.654752 (+ 737us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:12:37.654756 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:12:37.654759 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:12:37.654879 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:12:37.655295 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:12:37.655298 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:12:37.655300 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:12:37.655572 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:12:37.655680 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0807 18:12:37.656370 (+ 690us) server_negotiation.cc:299] Negotiation successful
0807 18:12:37.656535 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":375,"thread_start_us":265,"threads_started":1}
I20260807 18:12:37.657946 23215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:37.639013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:35307 (local address 127.18.154.3:44039)
0807 18:12:37.639915 (+ 902us) negotiation.cc:107] Waiting for socket to connect
0807 18:12:37.639983 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0807 18:12:37.641759 (+ 1776us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:12:37.642517 (+ 758us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:12:37.642526 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:12:37.643058 (+ 532us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:12:37.643666 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:12:37.643685 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:12:37.644797 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:12:37.644801 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:12:37.646323 (+ 1522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:12:37.646336 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:12:37.647116 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:12:37.647123 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:12:37.647363 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:37.648093 (+ 730us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:12:37.648115 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:12:37.650562 (+ 2447us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:12:37.654142 (+ 3580us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:12:37.654149 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:12:37.654164 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:12:37.654620 (+ 456us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:12:37.654977 (+ 357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:12:37.654981 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:12:37.654983 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:12:37.655174 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:12:37.655680 (+ 506us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:12:37.655686 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:12:37.656026 (+ 340us) client_negotiation.cc:769] Sending connection context
0807 18:12:37.656269 (+ 243us) client_negotiation.cc:240] Negotiation successful
0807 18:12:37.656668 (+ 399us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":552,"thread_start_us":320,"threads_started":1}
I20260807 18:12:37.659457 23211 heartbeater.cc:347] Connected to a master server at 127.18.154.62:35307
I20260807 18:12:37.659744 23211 heartbeater.cc:464] Registering TS with master...
I20260807 18:12:37.660530 23211 heartbeater.cc:511] Master 127.18.154.62:35307 requested a full tablet report, sending...
I20260807 18:12:37.662037 22557 ts_manager.cc:194] Registered new tserver with Master: af5ee4fe9a97406e96b54c59648bc526 (127.18.154.3:40383)
I20260807 18:12:37.662931 22557 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.154.3@KRBTEST.COM'} at 127.18.154.3:44039
I20260807 18:12:37.665118 19048 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260807 18:12:37.703323 23214 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:12:37.700649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39106 (local address 127.18.154.62:35307)
0807 18:12:37.700858 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0807 18:12:37.700863 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:12:37.701271 (+ 408us) server_negotiation.cc:407] Connection header received
0807 18:12:37.701429 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:12:37.701433 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:12:37.701501 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:12:37.701654 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:12:37.703010 (+ 1356us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39106: BlockingRecv error: recv got EOF from 127.0.0.1:39106 (error 108)
Metrics: {"server-negotiator.queue_time_us":89}
I20260807 18:12:37.706595 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 22653
W20260807 18:12:37.709486 19048 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 22653: Not authorized: Client connection negotiation failed: client connection to 127.18.154.1:36673: server requires authentication, but client does not have Kerberos credentials available
I20260807 18:12:37.709725 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 22653
I20260807 18:12:37.733858 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 22831
W20260807 18:12:37.736841 19048 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 22831: Not authorized: Client connection negotiation failed: client connection to 127.18.154.2:37925: server requires authentication, but client does not have Kerberos credentials available
I20260807 18:12:37.737008 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 22831
I20260807 18:12:37.762110 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 23029
W20260807 18:12:37.765232 19048 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 23029: Not authorized: Client connection negotiation failed: client connection to 127.18.154.3:40383: server requires authentication, but client does not have Kerberos credentials available
I20260807 18:12:37.765437 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 23029
I20260807 18:12:37.787420 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 22514
I20260807 18:12:37.788717 22606 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:12:37.892253 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 22514
2026-08-07T18:12:37Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (3935 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786126328808105-19048-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:12:38 dist-test-slave-gkw7 krb5kdc[23234](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:12:38 dist-test-slave-gkw7 krb5kdc[23234](info): set up 2 sockets
Aug 07 18:12:38 dist-test-slave-gkw7 krb5kdc[23234](info): commencing operation
krb5kdc: starting...
W20260807 18:12:39.174117 19048 mini_kdc.cc:121] Time spent starting KDC: real 1.234s user 0.002s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786126328808105-19048-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786126328808105-19048-0/krb5kdc/test-user.keytab.
Aug 07 18:12:39 dist-test-slave-gkw7 krb5kdc[23234](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126359, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:12:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:12:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.154.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.154.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.154.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.62.keytab.
Entry for principal kudu/127.18.154.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.62.keytab.
I20260807 18:12:39.403198 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786126328808105-19048-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786126328808105-19048-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786126328808105-19048-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.154.62:32785
--webserver_interface=127.18.154.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.62.keytab
--principal=kudu/127.18.154.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.154.20:46837
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.154.62:32785
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786126328808105-19048-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786126328808105-19048-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786126328808105-19048-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:12:39.744151 23253 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:12:39.744432 23253 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:12:39.744644 23253 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:12:39.751701 23253 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:12:39.751804 23253 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:12:39.751899 23253 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:12:39.752007 23253 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:12:39.752096 23253 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:12:39.761883 23253 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.154.20:46837
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786126328808105-19048-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786126328808105-19048-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.154.62:32785
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.62.keytab
--principal=kudu/127.18.154.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.154.62:32785
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.154.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786126328808105-19048-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:12:39.763280 23253 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:12:39.764669 23253 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:12:39.776932 23260 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:12:39.776909 23264 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:12:39.777122 23261 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:12:39.778448 23253 server_base.cc:1034] running on GCE node
I20260807 18:12:39.779273 23253 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:12:39.780653 23253 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:12:39.781863 23253 hybrid_clock.cc:648] HybridClock initialized: now 1786126359781799 us; error 51 us; skew 500 ppm
I20260807 18:12:39.782189 23253 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:12:39 dist-test-slave-gkw7 krb5kdc[23234](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126359, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:12:39.790524 23253 init.cc:377] Logged in from keytab as kudu/127.18.154.62@KRBTEST.COM (short username kudu)
I20260807 18:12:39.792254 23253 webserver.cc:460] Webserver started at http://127.18.154.62:43429/ using document root <none> and password file <none>
I20260807 18:12:39.793043 23253 fs_manager.cc:359] Metadata directory not provided
I20260807 18:12:39.793205 23253 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:12:39.793656 23253 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:12:39.797082 23253 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786126328808105-19048-0/minicluster-data/master-0/data/instance:
uuid: "dd943f04e8834d70b79bb954968fc9f3"
format_stamp: "Formatted at 2026-08-07 18:12:39 on dist-test-slave-gkw7"
server_key: "f5736c7371e704a1ae005d8fb5803a7e"
server_key_iv: "7f701a229bf16fdf8488a122da4874a9"
server_key_version: "encryptionkey@0"
I20260807 18:12:39.798040 23253 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786126328808105-19048-0/minicluster-data/master-0/wal/instance:
uuid: "dd943f04e8834d70b79bb954968fc9f3"
format_stamp: "Formatted at 2026-08-07 18:12:39 on dist-test-slave-gkw7"
server_key: "f5736c7371e704a1ae005d8fb5803a7e"
server_key_iv: "7f701a229bf16fdf8488a122da4874a9"
server_key_version: "encryptionkey@0"
I20260807 18:12:39.803813 23253 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 18:12:39.807873 23272 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:12:39.809442 23253 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260807 18:12:39.809710 23253 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786126328808105-19048-0/minicluster-data/master-0/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786126328808105-19048-0/minicluster-data/master-0/wal
uuid: "dd943f04e8834d70b79bb954968fc9f3"
format_stamp: "Formatted at 2026-08-07 18:12:39 on dist-test-slave-gkw7"
server_key: "f5736c7371e704a1ae005d8fb5803a7e"
server_key_iv: "7f701a229bf16fdf8488a122da4874a9"
server_key_version: "encryptionkey@0"
I20260807 18:12:39.809926 23253 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786126328808105-19048-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786126328808105-19048-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786126328808105-19048-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:12:39.828485 23253 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:12:39.833328 23253 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:12:39.833673 23253 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:12:39.852949 23253 rpc_server.cc:307] RPC server started. Bound to: 127.18.154.62:32785
I20260807 18:12:39.852998 23347 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.154.62:32785 every 8 connection(s)
I20260807 18:12:39.854447 23253 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
I20260807 18:12:39.855134 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 23253
I20260807 18:12:39.855489 19048 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786126328808105-19048-0/minicluster-data/master-0/wal/instance
I20260807 18:12:39.856288 19048 external_mini_cluster.cc:1428] Setting key df5946595bcd2e8b842a77a59faa1054
I20260807 18:12:39.859145 23348 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 07 18:12:39 dist-test-slave-gkw7 krb5kdc[23234](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126359, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.154.62@KRBTEST.COM
I20260807 18:12:39.871085 23348 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dd943f04e8834d70b79bb954968fc9f3: Bootstrap starting.
I20260807 18:12:39.874522 23348 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dd943f04e8834d70b79bb954968fc9f3: Neither blocks nor log segments found. Creating new log.
I20260807 18:12:39.875589 23348 log.cc:824] T 00000000000000000000000000000000 P dd943f04e8834d70b79bb954968fc9f3: Log is configured to *not* fsync() on all Append() calls
I20260807 18:12:39.878585 23348 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dd943f04e8834d70b79bb954968fc9f3: No bootstrap required, opened a new log
I20260807 18:12:39.879168 23354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:39.858245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37050 (local address 127.18.154.62:32785)
0807 18:12:39.859032 (+ 787us) server_negotiation.cc:206] Beginning negotiation
0807 18:12:39.859044 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 18:12:39.859084 (+ 40us) server_negotiation.cc:407] Connection header received
0807 18:12:39.860809 (+ 1725us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:12:39.860831 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:12:39.861204 (+ 373us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:12:39.861635 (+ 431us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:12:39.863188 (+ 1553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:39.864590 (+ 1402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:39.865934 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:39.866604 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:39.866819 (+ 215us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:39.871866 (+ 5047us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:12:39.871874 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:12:39.871888 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:12:39.871924 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:12:39.875092 (+ 3168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:12:39.875789 (+ 697us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:12:39.875796 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:12:39.875803 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:12:39.875964 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:12:39.876360 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:12:39.876364 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:12:39.876366 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:12:39.876913 (+ 547us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:12:39.877213 (+ 300us) server_negotiation.cc:1035] Waiting for connection context
0807 18:12:39.877748 (+ 535us) server_negotiation.cc:299] Negotiation successful
0807 18:12:39.878159 (+ 411us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":583,"thread_start_us":361,"threads_started":1}
I20260807 18:12:39.883278 19048 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:12:39.887059 23348 raft_consensus.cc:348] T 00000000000000000000000000000000 P dd943f04e8834d70b79bb954968fc9f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd943f04e8834d70b79bb954968fc9f3" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 32785 } }
I20260807 18:12:39.887439 23348 raft_consensus.cc:374] T 00000000000000000000000000000000 P dd943f04e8834d70b79bb954968fc9f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:12:39.887544 23348 raft_consensus.cc:729] T 00000000000000000000000000000000 P dd943f04e8834d70b79bb954968fc9f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd943f04e8834d70b79bb954968fc9f3, State: Initialized, Role: FOLLOWER
I20260807 18:12:39.888109 23348 consensus_queue.cc:260] T 00000000000000000000000000000000 P dd943f04e8834d70b79bb954968fc9f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd943f04e8834d70b79bb954968fc9f3" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 32785 } }
I20260807 18:12:39.888356 23348 raft_consensus.cc:388] T 00000000000000000000000000000000 P dd943f04e8834d70b79bb954968fc9f3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:12:39.888509 23348 raft_consensus.cc:482] T 00000000000000000000000000000000 P dd943f04e8834d70b79bb954968fc9f3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:12:39.888695 23348 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dd943f04e8834d70b79bb954968fc9f3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:12:39.890910 23348 raft_consensus.cc:504] T 00000000000000000000000000000000 P dd943f04e8834d70b79bb954968fc9f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd943f04e8834d70b79bb954968fc9f3" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 32785 } }
I20260807 18:12:39.891460 23348 leader_election.cc:302] T 00000000000000000000000000000000 P dd943f04e8834d70b79bb954968fc9f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dd943f04e8834d70b79bb954968fc9f3; no voters:
I20260807 18:12:39.892159 23348 leader_election.cc:288] T 00000000000000000000000000000000 P dd943f04e8834d70b79bb954968fc9f3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:12:39.892277 23358 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dd943f04e8834d70b79bb954968fc9f3 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:12:39.893633 23358 raft_consensus.cc:686] T 00000000000000000000000000000000 P dd943f04e8834d70b79bb954968fc9f3 [term 1 LEADER]: Becoming Leader. State: Replica: dd943f04e8834d70b79bb954968fc9f3, State: Running, Role: LEADER
I20260807 18:12:39.894186 23358 consensus_queue.cc:237] T 00000000000000000000000000000000 P dd943f04e8834d70b79bb954968fc9f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd943f04e8834d70b79bb954968fc9f3" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 32785 } }
I20260807 18:12:39.894601 23348 sys_catalog.cc:564] T 00000000000000000000000000000000 P dd943f04e8834d70b79bb954968fc9f3 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:12:39.897799 23360 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd943f04e8834d70b79bb954968fc9f3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dd943f04e8834d70b79bb954968fc9f3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd943f04e8834d70b79bb954968fc9f3" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 32785 } } }
I20260807 18:12:39.898193 23360 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd943f04e8834d70b79bb954968fc9f3 [sys.catalog]: This master's current role is: LEADER
I20260807 18:12:39.897243 23362 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd943f04e8834d70b79bb954968fc9f3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dd943f04e8834d70b79bb954968fc9f3. Latest consensus state: current_term: 1 leader_uuid: "dd943f04e8834d70b79bb954968fc9f3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd943f04e8834d70b79bb954968fc9f3" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 32785 } } }
I20260807 18:12:39.898700 23362 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd943f04e8834d70b79bb954968fc9f3 [sys.catalog]: This master's current role is: LEADER
I20260807 18:12:39.909874 23371 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:12:39.916517 23371 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:12:39.925737 23390 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260807 18:12:39.939975 23371 catalog_manager.cc:1347] Generated new cluster ID: b61e0cfa3dc04c7ba3d032892b32fe94
I20260807 18:12:39.940097 23371 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:12:39.969798 23371 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:12:39.970687 23354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:39.957940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37064 (local address 127.18.154.62:32785)
0807 18:12:39.958207 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0807 18:12:39.958217 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:12:39.958902 (+ 685us) server_negotiation.cc:407] Connection header received
0807 18:12:39.959222 (+ 320us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:12:39.959231 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:12:39.959327 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:12:39.959416 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:12:39.961303 (+ 1887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:39.962956 (+ 1653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:39.964971 (+ 2015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:39.965595 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:39.965717 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:39.968342 (+ 2625us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:12:39.968352 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:12:39.968359 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:12:39.969522 (+ 1163us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:12:39.969661 (+ 139us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:12:39.969698 (+ 37us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:12:39.969793 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0807 18:12:39.970069 (+ 276us) server_negotiation.cc:299] Negotiation successful
0807 18:12:39.970217 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260807 18:12:39.972259 23371 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:12:39.984611 23371 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dd943f04e8834d70b79bb954968fc9f3: Generated new TSK 0
I20260807 18:12:39.985762 23371 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:12:39.996465 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 23253
I20260807 18:12:39.998911 23341 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:12:40.103538 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 23253
2026-08-07T18:12:40Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (2212 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126328808105-19048-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:12:40 dist-test-slave-gkw7 krb5kdc[23406](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:12:40 dist-test-slave-gkw7 krb5kdc[23406](info): set up 2 sockets
Aug 07 18:12:40 dist-test-slave-gkw7 krb5kdc[23406](info): commencing operation
krb5kdc: starting...
W20260807 18:12:41.376366 19048 mini_kdc.cc:121] Time spent starting KDC: real 1.223s user 0.003s sys 0.019s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126328808105-19048-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126328808105-19048-0/krb5kdc/test-user.keytab.
Aug 07 18:12:41 dist-test-slave-gkw7 krb5kdc[23406](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126361, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:12:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:12:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.154.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.154.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.154.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.62.keytab.
Entry for principal kudu/127.18.154.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.62.keytab.
I20260807 18:12:41.612891 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126328808105-19048-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126328808105-19048-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126328808105-19048-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.154.62:40353
--webserver_interface=127.18.154.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.62.keytab
--principal=kudu/127.18.154.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.154.20:45061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.154.62:40353
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126328808105-19048-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126328808105-19048-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126328808105-19048-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:12:41.958532 23425 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:12:41.958868 23425 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:12:41.959034 23425 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:12:41.967752 23425 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:12:41.967877 23425 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:12:41.967991 23425 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:12:41.968103 23425 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:12:41.968187 23425 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:12:41.978439 23425 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.154.20:45061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126328808105-19048-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126328808105-19048-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.154.62:40353
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.62.keytab
--principal=kudu/127.18.154.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.154.62:40353
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.154.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126328808105-19048-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:12:41.979938 23425 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:12:41.981352 23425 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:12:41.994472 23433 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:12:41.994547 23438 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:12:41.994500 23435 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:12:41.995249 23425 server_base.cc:1034] running on GCE node
I20260807 18:12:41.996263 23425 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:12:41.997970 23425 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:12:41.999233 23425 hybrid_clock.cc:648] HybridClock initialized: now 1786126361999159 us; error 64 us; skew 500 ppm
I20260807 18:12:41.999624 23425 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:12:41 dist-test-slave-gkw7 krb5kdc[23406](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126361, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:12:42.003178 23425 init.cc:377] Logged in from keytab as kudu/127.18.154.62@KRBTEST.COM (short username kudu)
I20260807 18:12:42.004788 23425 webserver.cc:460] Webserver started at http://127.18.154.62:37963/ using document root <none> and password file <none>
I20260807 18:12:42.005784 23425 fs_manager.cc:359] Metadata directory not provided
I20260807 18:12:42.006007 23425 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:12:42.006414 23425 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:12:42.010356 23425 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126328808105-19048-0/minicluster-data/master-0/data/instance:
uuid: "27935d51332d42179bad8dd235d68e30"
format_stamp: "Formatted at 2026-08-07 18:12:42 on dist-test-slave-gkw7"
server_key: "7016f89660ca90a9a2fc0d2c91ed7d7d"
server_key_iv: "61cfbca80fa92c1dff05a25e9b5cddc0"
server_key_version: "encryptionkey@0"
I20260807 18:12:42.011102 23425 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126328808105-19048-0/minicluster-data/master-0/wal/instance:
uuid: "27935d51332d42179bad8dd235d68e30"
format_stamp: "Formatted at 2026-08-07 18:12:42 on dist-test-slave-gkw7"
server_key: "7016f89660ca90a9a2fc0d2c91ed7d7d"
server_key_iv: "61cfbca80fa92c1dff05a25e9b5cddc0"
server_key_version: "encryptionkey@0"
I20260807 18:12:42.017045 23425 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260807 18:12:42.021237 23448 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:12:42.023147 23425 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 18:12:42.023343 23425 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126328808105-19048-0/minicluster-data/master-0/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126328808105-19048-0/minicluster-data/master-0/wal
uuid: "27935d51332d42179bad8dd235d68e30"
format_stamp: "Formatted at 2026-08-07 18:12:42 on dist-test-slave-gkw7"
server_key: "7016f89660ca90a9a2fc0d2c91ed7d7d"
server_key_iv: "61cfbca80fa92c1dff05a25e9b5cddc0"
server_key_version: "encryptionkey@0"
I20260807 18:12:42.023653 23425 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126328808105-19048-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126328808105-19048-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126328808105-19048-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:12:42.054544 23425 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:12:42.059834 23425 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:12:42.060189 23425 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:12:42.082561 23425 rpc_server.cc:307] RPC server started. Bound to: 127.18.154.62:40353
I20260807 18:12:42.082612 23523 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.154.62:40353 every 8 connection(s)
I20260807 18:12:42.084184 23425 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
I20260807 18:12:42.087642 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 23425
I20260807 18:12:42.087903 19048 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126328808105-19048-0/minicluster-data/master-0/wal/instance
I20260807 18:12:42.088722 19048 external_mini_cluster.cc:1428] Setting key 5a3cd2bc4ae0ba8388d62706bbc75757
I20260807 18:12:42.089133 23524 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:12:42.104526 23524 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 27935d51332d42179bad8dd235d68e30: Bootstrap starting.
Aug 07 18:12:42 dist-test-slave-gkw7 krb5kdc[23406](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126361, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.154.62@KRBTEST.COM
I20260807 18:12:42.107810 23524 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 27935d51332d42179bad8dd235d68e30: Neither blocks nor log segments found. Creating new log.
I20260807 18:12:42.108884 23524 log.cc:824] T 00000000000000000000000000000000 P 27935d51332d42179bad8dd235d68e30: Log is configured to *not* fsync() on all Append() calls
I20260807 18:12:42.111918 23524 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 27935d51332d42179bad8dd235d68e30: No bootstrap required, opened a new log
I20260807 18:12:42.114655 23528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:42.091043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42064 (local address 127.18.154.62:40353)
0807 18:12:42.091932 (+ 889us) server_negotiation.cc:206] Beginning negotiation
0807 18:12:42.091945 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 18:12:42.091993 (+ 48us) server_negotiation.cc:407] Connection header received
0807 18:12:42.094252 (+ 2259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:12:42.094278 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:12:42.094717 (+ 439us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:12:42.095087 (+ 370us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:12:42.096855 (+ 1768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:42.098376 (+ 1521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:42.100166 (+ 1790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:42.100969 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:42.101138 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:42.106350 (+ 5212us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:12:42.106359 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:12:42.106376 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:12:42.106426 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:12:42.110105 (+ 3679us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:12:42.110884 (+ 779us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:12:42.110892 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:12:42.110899 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:12:42.111079 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:12:42.111528 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:12:42.111535 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:12:42.111539 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:12:42.112173 (+ 634us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:12:42.112568 (+ 395us) server_negotiation.cc:1035] Waiting for connection context
0807 18:12:42.112999 (+ 431us) server_negotiation.cc:299] Negotiation successful
0807 18:12:42.113468 (+ 469us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":657,"thread_start_us":401,"threads_started":1}
I20260807 18:12:42.120630 23524 raft_consensus.cc:348] T 00000000000000000000000000000000 P 27935d51332d42179bad8dd235d68e30 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27935d51332d42179bad8dd235d68e30" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 40353 } }
I20260807 18:12:42.121107 23524 raft_consensus.cc:374] T 00000000000000000000000000000000 P 27935d51332d42179bad8dd235d68e30 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:12:42.121250 23524 raft_consensus.cc:729] T 00000000000000000000000000000000 P 27935d51332d42179bad8dd235d68e30 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27935d51332d42179bad8dd235d68e30, State: Initialized, Role: FOLLOWER
I20260807 18:12:42.121941 23524 consensus_queue.cc:260] T 00000000000000000000000000000000 P 27935d51332d42179bad8dd235d68e30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27935d51332d42179bad8dd235d68e30" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 40353 } }
I20260807 18:12:42.122206 23524 raft_consensus.cc:388] T 00000000000000000000000000000000 P 27935d51332d42179bad8dd235d68e30 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:12:42.122357 23524 raft_consensus.cc:482] T 00000000000000000000000000000000 P 27935d51332d42179bad8dd235d68e30 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:12:42.122534 23524 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 27935d51332d42179bad8dd235d68e30 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:12:42.125111 23524 raft_consensus.cc:504] T 00000000000000000000000000000000 P 27935d51332d42179bad8dd235d68e30 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27935d51332d42179bad8dd235d68e30" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 40353 } }
I20260807 18:12:42.125696 23524 leader_election.cc:302] T 00000000000000000000000000000000 P 27935d51332d42179bad8dd235d68e30 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 27935d51332d42179bad8dd235d68e30; no voters:
I20260807 18:12:42.126319 23524 leader_election.cc:288] T 00000000000000000000000000000000 P 27935d51332d42179bad8dd235d68e30 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:12:42.126470 23531 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 27935d51332d42179bad8dd235d68e30 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:12:42.127206 23531 raft_consensus.cc:686] T 00000000000000000000000000000000 P 27935d51332d42179bad8dd235d68e30 [term 1 LEADER]: Becoming Leader. State: Replica: 27935d51332d42179bad8dd235d68e30, State: Running, Role: LEADER
I20260807 18:12:42.127947 23531 consensus_queue.cc:237] T 00000000000000000000000000000000 P 27935d51332d42179bad8dd235d68e30 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27935d51332d42179bad8dd235d68e30" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 40353 } }
I20260807 18:12:42.128541 23524 sys_catalog.cc:564] T 00000000000000000000000000000000 P 27935d51332d42179bad8dd235d68e30 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:12:42.131525 23532 sys_catalog.cc:455] T 00000000000000000000000000000000 P 27935d51332d42179bad8dd235d68e30 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "27935d51332d42179bad8dd235d68e30" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27935d51332d42179bad8dd235d68e30" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 40353 } } }
I20260807 18:12:42.131637 23533 sys_catalog.cc:455] T 00000000000000000000000000000000 P 27935d51332d42179bad8dd235d68e30 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 27935d51332d42179bad8dd235d68e30. Latest consensus state: current_term: 1 leader_uuid: "27935d51332d42179bad8dd235d68e30" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27935d51332d42179bad8dd235d68e30" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 40353 } } }
I20260807 18:12:42.132025 23533 sys_catalog.cc:458] T 00000000000000000000000000000000 P 27935d51332d42179bad8dd235d68e30 [sys.catalog]: This master's current role is: LEADER
I20260807 18:12:42.132030 23532 sys_catalog.cc:458] T 00000000000000000000000000000000 P 27935d51332d42179bad8dd235d68e30 [sys.catalog]: This master's current role is: LEADER
I20260807 18:12:42.134678 23538 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:12:42.140334 23538 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:12:42.149935 23538 catalog_manager.cc:1347] Generated new cluster ID: cf05baeac0f74fa39000581e049b9bc0
I20260807 18:12:42.150069 23538 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:12:42.164454 23538 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:12:42.165907 23538 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:12:42.173811 23538 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 27935d51332d42179bad8dd235d68e30: Generated new TSK 0
I20260807 18:12:42.174917 23538 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.154.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.154.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.154.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.1.keytab.
Entry for principal kudu/127.18.154.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.1.keytab.
I20260807 18:12:42.242995 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126328808105-19048-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126328808105-19048-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126328808105-19048-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126328808105-19048-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.154.1:0
--local_ip_for_outbound_sockets=127.18.154.1
--webserver_interface=127.18.154.1
--webserver_port=0
--tserver_master_addrs=127.18.154.62:40353
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.1.keytab
--principal=kudu/127.18.154.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.154.20:45061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126328808105-19048-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126328808105-19048-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126328808105-19048-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:12:42.598851 23557 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:12:42.599159 23557 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:12:42.599354 23557 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:12:42.606884 23557 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:12:42.607072 23557 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:12:42.607249 23557 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.154.1
I20260807 18:12:42.617692 23557 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.154.20:45061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126328808105-19048-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126328808105-19048-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.1.keytab
--principal=kudu/127.18.154.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.154.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126328808105-19048-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.154.1
--webserver_port=0
--tserver_master_addrs=127.18.154.62:40353
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.154.1
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126328808105-19048-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:12:42.619124 23557 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:12:42.620517 23557 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:12:42.634987 23566 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:12:42.635035 23565 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:12:42.635156 23569 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:12:42.636459 23557 server_base.cc:1034] running on GCE node
I20260807 18:12:42.637089 23557 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:12:42.638203 23557 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:12:42.639433 23557 hybrid_clock.cc:648] HybridClock initialized: now 1786126362639384 us; error 30 us; skew 500 ppm
I20260807 18:12:42.639838 23557 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:12:42 dist-test-slave-gkw7 krb5kdc[23406](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126362, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:12:42.643230 23557 init.cc:377] Logged in from keytab as kudu/127.18.154.1@KRBTEST.COM (short username kudu)
I20260807 18:12:42.644848 23557 webserver.cc:460] Webserver started at http://127.18.154.1:46377/ using document root <none> and password file <none>
I20260807 18:12:42.645665 23557 fs_manager.cc:359] Metadata directory not provided
I20260807 18:12:42.645825 23557 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:12:42.646157 23557 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:12:42.649346 23557 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126328808105-19048-0/minicluster-data/ts-0/data/instance:
uuid: "050403c3fc6b452187937d611533058d"
format_stamp: "Formatted at 2026-08-07 18:12:42 on dist-test-slave-gkw7"
server_key: "4c518e5ba9ac65c90c6a364f422ca17d"
server_key_iv: "6140dd5449c8f50510e05d3e8e5dd46f"
server_key_version: "encryptionkey@0"
I20260807 18:12:42.650252 23557 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126328808105-19048-0/minicluster-data/ts-0/wal/instance:
uuid: "050403c3fc6b452187937d611533058d"
format_stamp: "Formatted at 2026-08-07 18:12:42 on dist-test-slave-gkw7"
server_key: "4c518e5ba9ac65c90c6a364f422ca17d"
server_key_iv: "6140dd5449c8f50510e05d3e8e5dd46f"
server_key_version: "encryptionkey@0"
I20260807 18:12:42.655851 23557 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260807 18:12:42.659700 23579 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:12:42.661185 23557 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 18:12:42.661375 23557 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126328808105-19048-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126328808105-19048-0/minicluster-data/ts-0/wal
uuid: "050403c3fc6b452187937d611533058d"
format_stamp: "Formatted at 2026-08-07 18:12:42 on dist-test-slave-gkw7"
server_key: "4c518e5ba9ac65c90c6a364f422ca17d"
server_key_iv: "6140dd5449c8f50510e05d3e8e5dd46f"
server_key_version: "encryptionkey@0"
I20260807 18:12:42.661667 23557 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126328808105-19048-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126328808105-19048-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126328808105-19048-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:12:42.688315 23557 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:12:42.692651 23557 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:12:42.692936 23557 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:12:42.693990 23557 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:12:42.695372 23557 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:12:42.695447 23557 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:12:42.695534 23557 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:12:42.695578 23557 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:12:42.731197 23557 rpc_server.cc:307] RPC server started. Bound to: 127.18.154.1:36457
I20260807 18:12:42.731245 23732 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.154.1:36457 every 8 connection(s)
I20260807 18:12:42.732601 23557 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126328808105-19048-0/minicluster-data/ts-0/data/info.pb
I20260807 18:12:42.739821 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 23557
I20260807 18:12:42.740072 19048 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126328808105-19048-0/minicluster-data/ts-0/wal/instance
I20260807 18:12:42.740767 19048 external_mini_cluster.cc:1428] Setting key 667ba47183864fe326401c6568068b57
Aug 07 18:12:42 dist-test-slave-gkw7 krb5kdc[23406](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126362, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.1@KRBTEST.COM for kudu/127.18.154.62@KRBTEST.COM
I20260807 18:12:42.756119 23738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:42.735328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.1:59365 (local address 127.18.154.62:40353)
0807 18:12:42.735867 (+ 539us) server_negotiation.cc:206] Beginning negotiation
0807 18:12:42.735874 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:12:42.736949 (+ 1075us) server_negotiation.cc:407] Connection header received
0807 18:12:42.738861 (+ 1912us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:12:42.738865 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:12:42.738942 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:12:42.739045 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:12:42.740810 (+ 1765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:42.741871 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:42.744458 (+ 2587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:42.744860 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:42.744966 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:42.749394 (+ 4428us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:12:42.749398 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:12:42.749402 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:12:42.749436 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:12:42.752712 (+ 3276us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:12:42.753439 (+ 727us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:12:42.753447 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:12:42.753454 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:12:42.753644 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:12:42.754078 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:12:42.754085 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:12:42.754091 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:12:42.754471 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:12:42.754618 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0807 18:12:42.755351 (+ 733us) server_negotiation.cc:299] Negotiation successful
0807 18:12:42.755617 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":391,"thread_start_us":252,"threads_started":1}
I20260807 18:12:42.756793 23739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:42.735945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:40353 (local address 127.18.154.1:59365)
0807 18:12:42.736779 (+ 834us) negotiation.cc:107] Waiting for socket to connect
0807 18:12:42.736818 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 18:12:42.738580 (+ 1762us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:12:42.739275 (+ 695us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:12:42.739284 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:12:42.739872 (+ 588us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:12:42.740608 (+ 736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:12:42.740628 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:12:42.742037 (+ 1409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:12:42.742045 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:12:42.744260 (+ 2215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:12:42.744280 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:12:42.745112 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:12:42.745122 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:12:42.745480 (+ 358us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:42.746306 (+ 826us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:12:42.746359 (+ 53us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:12:42.749150 (+ 2791us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:12:42.752862 (+ 3712us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:12:42.752869 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:12:42.752883 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:12:42.753313 (+ 430us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:12:42.753764 (+ 451us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:12:42.753768 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:12:42.753770 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:12:42.753962 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:12:42.754607 (+ 645us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:12:42.754613 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:12:42.754951 (+ 338us) client_negotiation.cc:769] Sending connection context
0807 18:12:42.755191 (+ 240us) client_negotiation.cc:240] Negotiation successful
0807 18:12:42.755599 (+ 408us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":547,"thread_start_us":299,"threads_started":1}
I20260807 18:12:42.758569 23733 heartbeater.cc:347] Connected to a master server at 127.18.154.62:40353
I20260807 18:12:42.758970 23733 heartbeater.cc:464] Registering TS with master...
I20260807 18:12:42.759922 23733 heartbeater.cc:511] Master 127.18.154.62:40353 requested a full tablet report, sending...
I20260807 18:12:42.762508 23470 ts_manager.cc:194] Registered new tserver with Master: 050403c3fc6b452187937d611533058d (127.18.154.1:36457)
I20260807 18:12:42.764053 23470 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.154.1@KRBTEST.COM'} at 127.18.154.1:59365
WARNING: no policy specified for kudu/127.18.154.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.154.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.154.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.2.keytab.
Entry for principal kudu/127.18.154.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.2.keytab.
I20260807 18:12:42.797389 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126328808105-19048-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126328808105-19048-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126328808105-19048-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126328808105-19048-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.154.2:0
--local_ip_for_outbound_sockets=127.18.154.2
--webserver_interface=127.18.154.2
--webserver_port=0
--tserver_master_addrs=127.18.154.62:40353
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.2.keytab
--principal=kudu/127.18.154.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.154.20:45061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126328808105-19048-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126328808105-19048-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126328808105-19048-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:12:43.146853 23742 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:12:43.147233 23742 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:12:43.147439 23742 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:12:43.154886 23742 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:12:43.155078 23742 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:12:43.155282 23742 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.154.2
I20260807 18:12:43.165740 23742 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.154.20:45061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126328808105-19048-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126328808105-19048-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.2.keytab
--principal=kudu/127.18.154.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.154.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126328808105-19048-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.154.2
--webserver_port=0
--tserver_master_addrs=127.18.154.62:40353
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.154.2
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126328808105-19048-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:12:43.167255 23742 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:12:43.168658 23742 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:12:43.182873 23753 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:12:43.182988 23749 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:12:43.183440 23748 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:12:43.183832 23742 server_base.cc:1034] running on GCE node
I20260807 18:12:43.184527 23742 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:12:43.185580 23742 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:12:43.186893 23742 hybrid_clock.cc:648] HybridClock initialized: now 1786126363186818 us; error 47 us; skew 500 ppm
I20260807 18:12:43.187299 23742 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:12:43 dist-test-slave-gkw7 krb5kdc[23406](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126363, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:12:43.190897 23742 init.cc:377] Logged in from keytab as kudu/127.18.154.2@KRBTEST.COM (short username kudu)
I20260807 18:12:43.192448 23742 webserver.cc:460] Webserver started at http://127.18.154.2:39391/ using document root <none> and password file <none>
I20260807 18:12:43.193234 23742 fs_manager.cc:359] Metadata directory not provided
I20260807 18:12:43.193379 23742 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:12:43.193794 23742 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:12:43.197117 23742 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126328808105-19048-0/minicluster-data/ts-1/data/instance:
uuid: "3b4a1f63b055478988deeb73bb6c1312"
format_stamp: "Formatted at 2026-08-07 18:12:43 on dist-test-slave-gkw7"
server_key: "221c3ea0ea457bc3949b97ab67bec372"
server_key_iv: "34db690f2c98bdaed7450bd68bb5bca3"
server_key_version: "encryptionkey@0"
I20260807 18:12:43.198045 23742 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126328808105-19048-0/minicluster-data/ts-1/wal/instance:
uuid: "3b4a1f63b055478988deeb73bb6c1312"
format_stamp: "Formatted at 2026-08-07 18:12:43 on dist-test-slave-gkw7"
server_key: "221c3ea0ea457bc3949b97ab67bec372"
server_key_iv: "34db690f2c98bdaed7450bd68bb5bca3"
server_key_version: "encryptionkey@0"
I20260807 18:12:43.203788 23742 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 18:12:43.207630 23765 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:12:43.209106 23742 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 18:12:43.209335 23742 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126328808105-19048-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126328808105-19048-0/minicluster-data/ts-1/wal
uuid: "3b4a1f63b055478988deeb73bb6c1312"
format_stamp: "Formatted at 2026-08-07 18:12:43 on dist-test-slave-gkw7"
server_key: "221c3ea0ea457bc3949b97ab67bec372"
server_key_iv: "34db690f2c98bdaed7450bd68bb5bca3"
server_key_version: "encryptionkey@0"
I20260807 18:12:43.209580 23742 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126328808105-19048-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126328808105-19048-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126328808105-19048-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:12:43.226563 23742 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:12:43.231513 23742 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:12:43.231840 23742 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:12:43.232892 23742 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:12:43.234354 23742 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:12:43.234458 23742 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:12:43.234617 23742 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:12:43.234689 23742 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:12:43.269918 23742 rpc_server.cc:307] RPC server started. Bound to: 127.18.154.2:42845
I20260807 18:12:43.269968 23935 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.154.2:42845 every 8 connection(s)
I20260807 18:12:43.271338 23742 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126328808105-19048-0/minicluster-data/ts-1/data/info.pb
I20260807 18:12:43.281086 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 23742
I20260807 18:12:43.281320 19048 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126328808105-19048-0/minicluster-data/ts-1/wal/instance
I20260807 18:12:43.281991 19048 external_mini_cluster.cc:1428] Setting key 0836148ac06f51e9beb1bd814d94e958
Aug 07 18:12:43 dist-test-slave-gkw7 krb5kdc[23406](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126363, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.2@KRBTEST.COM for kudu/127.18.154.62@KRBTEST.COM
I20260807 18:12:43.299990 23940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:43.274502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.2:56453 (local address 127.18.154.62:40353)
0807 18:12:43.282682 (+ 8180us) server_negotiation.cc:206] Beginning negotiation
0807 18:12:43.282689 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:12:43.282712 (+ 23us) server_negotiation.cc:407] Connection header received
0807 18:12:43.282783 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:12:43.282790 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:12:43.282885 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:12:43.283004 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:12:43.285049 (+ 2045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:43.286347 (+ 1298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:43.287985 (+ 1638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:43.288759 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:43.288918 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:43.292884 (+ 3966us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:12:43.292891 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:12:43.292895 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:12:43.292946 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:12:43.296055 (+ 3109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:12:43.297268 (+ 1213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:12:43.297272 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:12:43.297275 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:12:43.297414 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:12:43.298039 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:12:43.298042 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:12:43.298044 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:12:43.298349 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:12:43.298468 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0807 18:12:43.299511 (+ 1043us) server_negotiation.cc:299] Negotiation successful
0807 18:12:43.299694 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8034,"thread_start_us":7910,"threads_started":1}
I20260807 18:12:43.301955 23942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:43.275248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:40353 (local address 127.18.154.2:56453)
0807 18:12:43.278267 (+ 3019us) negotiation.cc:107] Waiting for socket to connect
0807 18:12:43.278347 (+ 80us) client_negotiation.cc:174] Beginning negotiation
0807 18:12:43.281728 (+ 3381us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:12:43.283448 (+ 1720us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:12:43.283458 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:12:43.284168 (+ 710us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:12:43.284848 (+ 680us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:12:43.284867 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:12:43.286513 (+ 1646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:12:43.286517 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:12:43.287818 (+ 1301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:12:43.287831 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:12:43.289020 (+ 1189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:12:43.289026 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:12:43.289319 (+ 293us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:43.290227 (+ 908us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:12:43.290252 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:12:43.292603 (+ 2351us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:12:43.296228 (+ 3625us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:12:43.296242 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:12:43.296266 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:12:43.297095 (+ 829us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:12:43.297570 (+ 475us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:12:43.297578 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:12:43.297585 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:12:43.297890 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:12:43.298505 (+ 615us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:12:43.298518 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:12:43.299078 (+ 560us) client_negotiation.cc:769] Sending connection context
0807 18:12:43.299435 (+ 357us) client_negotiation.cc:240] Negotiation successful
0807 18:12:43.300157 (+ 722us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2560,"thread_start_us":2238,"threads_started":1}
I20260807 18:12:43.303627 23938 heartbeater.cc:347] Connected to a master server at 127.18.154.62:40353
I20260807 18:12:43.303980 23938 heartbeater.cc:464] Registering TS with master...
I20260807 18:12:43.304790 23938 heartbeater.cc:511] Master 127.18.154.62:40353 requested a full tablet report, sending...
I20260807 18:12:43.306478 23470 ts_manager.cc:194] Registered new tserver with Master: 3b4a1f63b055478988deeb73bb6c1312 (127.18.154.2:42845)
I20260807 18:12:43.307364 23470 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.154.2@KRBTEST.COM'} at 127.18.154.2:56453
WARNING: no policy specified for kudu/127.18.154.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.154.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.154.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.3.keytab.
Entry for principal kudu/127.18.154.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.3.keytab.
I20260807 18:12:43.339316 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126328808105-19048-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126328808105-19048-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126328808105-19048-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126328808105-19048-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.154.3:0
--local_ip_for_outbound_sockets=127.18.154.3
--webserver_interface=127.18.154.3
--webserver_port=0
--tserver_master_addrs=127.18.154.62:40353
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.3.keytab
--principal=kudu/127.18.154.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.154.20:45061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126328808105-19048-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126328808105-19048-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126328808105-19048-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:12:43.681185 23947 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:12:43.681528 23947 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:12:43.681715 23947 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:12:43.689042 23947 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:12:43.689214 23947 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:12:43.689388 23947 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.154.3
I20260807 18:12:43.699273 23947 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.154.20:45061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126328808105-19048-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126328808105-19048-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.3.keytab
--principal=kudu/127.18.154.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.154.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126328808105-19048-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.154.3
--webserver_port=0
--tserver_master_addrs=127.18.154.62:40353
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.154.3
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126328808105-19048-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:12:43.700713 23947 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:12:43.702234 23947 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:12:43.716050 23954 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:12:43.716081 23957 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:12:43.716107 23955 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:12:43.717326 23947 server_base.cc:1034] running on GCE node
I20260807 18:12:43.718180 23947 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:12:43.719468 23947 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:12:43.720726 23947 hybrid_clock.cc:648] HybridClock initialized: now 1786126363720709 us; error 84 us; skew 500 ppm
I20260807 18:12:43.721086 23947 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:12:43 dist-test-slave-gkw7 krb5kdc[23406](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126363, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:12:43.724483 23947 init.cc:377] Logged in from keytab as kudu/127.18.154.3@KRBTEST.COM (short username kudu)
I20260807 18:12:43.726202 23947 webserver.cc:460] Webserver started at http://127.18.154.3:38801/ using document root <none> and password file <none>
I20260807 18:12:43.726904 23947 fs_manager.cc:359] Metadata directory not provided
I20260807 18:12:43.727010 23947 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:12:43.727289 23947 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:12:43.730993 23947 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126328808105-19048-0/minicluster-data/ts-2/data/instance:
uuid: "e8176b29dd124707aebb624e044909e0"
format_stamp: "Formatted at 2026-08-07 18:12:43 on dist-test-slave-gkw7"
server_key: "66cb3a818b0ebca18598439b6e124cd8"
server_key_iv: "f8686bef9d0655a301fea1bb6839e5e2"
server_key_version: "encryptionkey@0"
I20260807 18:12:43.732110 23947 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126328808105-19048-0/minicluster-data/ts-2/wal/instance:
uuid: "e8176b29dd124707aebb624e044909e0"
format_stamp: "Formatted at 2026-08-07 18:12:43 on dist-test-slave-gkw7"
server_key: "66cb3a818b0ebca18598439b6e124cd8"
server_key_iv: "f8686bef9d0655a301fea1bb6839e5e2"
server_key_version: "encryptionkey@0"
I20260807 18:12:43.738739 23947 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260807 18:12:43.742508 23967 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:12:43.744091 23947 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 18:12:43.744287 23947 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126328808105-19048-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126328808105-19048-0/minicluster-data/ts-2/wal
uuid: "e8176b29dd124707aebb624e044909e0"
format_stamp: "Formatted at 2026-08-07 18:12:43 on dist-test-slave-gkw7"
server_key: "66cb3a818b0ebca18598439b6e124cd8"
server_key_iv: "f8686bef9d0655a301fea1bb6839e5e2"
server_key_version: "encryptionkey@0"
I20260807 18:12:43.744514 23947 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126328808105-19048-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126328808105-19048-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126328808105-19048-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:12:43.761168 23947 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:12:43.765898 23947 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:12:43.766276 23947 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:12:43.767343 23947 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:12:43.767607 23733 heartbeater.cc:503] Master 127.18.154.62:40353 was elected leader, sending a full tablet report...
I20260807 18:12:43.769100 23947 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:12:43.769220 23947 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:12:43.769397 23947 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:12:43.769491 23947 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:12:43.807632 23947 rpc_server.cc:307] RPC server started. Bound to: 127.18.154.3:39683
I20260807 18:12:43.807669 24127 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.154.3:39683 every 8 connection(s)
I20260807 18:12:43.809170 23947 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126328808105-19048-0/minicluster-data/ts-2/data/info.pb
I20260807 18:12:43.812662 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 23947
I20260807 18:12:43.812881 19048 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786126328808105-19048-0/minicluster-data/ts-2/wal/instance
I20260807 18:12:43.813421 19048 external_mini_cluster.cc:1428] Setting key 4ce110aba124968bafb269b1443866f2
Aug 07 18:12:43 dist-test-slave-gkw7 krb5kdc[23406](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126363, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.3@KRBTEST.COM for kudu/127.18.154.62@KRBTEST.COM
I20260807 18:12:43.832160 24132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:43.812059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.3:48317 (local address 127.18.154.62:40353)
0807 18:12:43.812610 (+ 551us) server_negotiation.cc:206] Beginning negotiation
0807 18:12:43.812617 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:12:43.814154 (+ 1537us) server_negotiation.cc:407] Connection header received
0807 18:12:43.816625 (+ 2471us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:12:43.816630 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:12:43.816755 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:12:43.816884 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:12:43.818583 (+ 1699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:43.819433 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:43.821047 (+ 1614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:43.821458 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:43.821620 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:43.825356 (+ 3736us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:12:43.825361 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:12:43.825364 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:12:43.825405 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:12:43.828289 (+ 2884us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:12:43.829445 (+ 1156us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:12:43.829452 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:12:43.829456 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:12:43.829680 (+ 224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:12:43.830267 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:12:43.830272 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:12:43.830276 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:12:43.830576 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:12:43.830749 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0807 18:12:43.831513 (+ 764us) server_negotiation.cc:299] Negotiation successful
0807 18:12:43.831771 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":406,"thread_start_us":282,"threads_started":1}
I20260807 18:12:43.833278 24134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:43.812782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:40353 (local address 127.18.154.3:48317)
0807 18:12:43.813955 (+ 1173us) negotiation.cc:107] Waiting for socket to connect
0807 18:12:43.814012 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0807 18:12:43.816353 (+ 2341us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:12:43.817147 (+ 794us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:12:43.817156 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:12:43.817784 (+ 628us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:12:43.818402 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:12:43.818421 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:12:43.819607 (+ 1186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:12:43.819611 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:12:43.820890 (+ 1279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:12:43.820903 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:12:43.821749 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:12:43.821758 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:12:43.822029 (+ 271us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:43.822756 (+ 727us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:12:43.822782 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:12:43.825112 (+ 2330us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:12:43.828505 (+ 3393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:12:43.828515 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:12:43.828535 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:12:43.829282 (+ 747us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:12:43.829812 (+ 530us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:12:43.829818 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:12:43.829822 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:12:43.830153 (+ 331us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:12:43.830719 (+ 566us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:12:43.830728 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:12:43.831114 (+ 386us) client_negotiation.cc:769] Sending connection context
0807 18:12:43.831406 (+ 292us) client_negotiation.cc:240] Negotiation successful
0807 18:12:43.831983 (+ 577us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":805,"thread_start_us":355,"threads_started":1}
I20260807 18:12:43.834702 24129 heartbeater.cc:347] Connected to a master server at 127.18.154.62:40353
I20260807 18:12:43.835036 24129 heartbeater.cc:464] Registering TS with master...
I20260807 18:12:43.835863 24129 heartbeater.cc:511] Master 127.18.154.62:40353 requested a full tablet report, sending...
I20260807 18:12:43.837321 23470 ts_manager.cc:194] Registered new tserver with Master: e8176b29dd124707aebb624e044909e0 (127.18.154.3:39683)
I20260807 18:12:43.838201 23470 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.154.3@KRBTEST.COM'} at 127.18.154.3:48317
I20260807 18:12:43.842447 19048 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 18:12:43 dist-test-slave-gkw7 krb5kdc[23406](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126363, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 07 18:12:43 dist-test-slave-gkw7 krb5kdc[23406](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126363, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.18.154.62@KRBTEST.COM
I20260807 18:12:43.896808 24132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:43.882322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42068 (local address 127.18.154.62:40353)
0807 18:12:43.882515 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0807 18:12:43.882519 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:12:43.882966 (+ 447us) server_negotiation.cc:407] Connection header received
0807 18:12:43.883123 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:12:43.883127 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:12:43.883208 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:12:43.883306 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:12:43.884698 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:43.885438 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:43.886787 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:43.887192 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:43.887300 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:43.891668 (+ 4368us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:12:43.891672 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:12:43.891676 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:12:43.891711 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:12:43.894324 (+ 2613us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:12:43.895014 (+ 690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:12:43.895017 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:12:43.895020 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:12:43.895137 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:12:43.895535 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:12:43.895538 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:12:43.895540 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:12:43.895879 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:12:43.896071 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0807 18:12:43.896368 (+ 297us) server_negotiation.cc:299] Negotiation successful
0807 18:12:43.896568 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
W20260807 18:12:43.897599 23470 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:42068
I20260807 18:12:43.899322 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 23557
Aug 07 18:12:43 dist-test-slave-gkw7 krb5kdc[23406](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126363, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.18.154.1@KRBTEST.COM
I20260807 18:12:43.915967 24145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:43.900547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44832 (local address 127.18.154.1:36457)
0807 18:12:43.901199 (+ 652us) server_negotiation.cc:206] Beginning negotiation
0807 18:12:43.901206 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:12:43.901231 (+ 25us) server_negotiation.cc:407] Connection header received
0807 18:12:43.901290 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:12:43.901296 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:12:43.901498 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:12:43.901722 (+ 224us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:12:43.903010 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:43.904007 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:43.905291 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:43.905932 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:43.906031 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:43.910302 (+ 4271us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:12:43.910314 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:12:43.910328 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:12:43.910366 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:12:43.913170 (+ 2804us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:12:43.913932 (+ 762us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:12:43.913945 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:12:43.913948 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:12:43.914158 (+ 210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:12:43.914510 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:12:43.914514 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:12:43.914516 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:12:43.914925 (+ 409us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:12:43.915195 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0807 18:12:43.915364 (+ 169us) server_negotiation.cc:299] Negotiation successful
0807 18:12:43.915557 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":491,"thread_start_us":341,"threads_started":1}
W20260807 18:12:43.917295 23727 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:44832
W20260807 18:12:43.918517 19048 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 23557: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 18:12:43.918685 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 23557
I20260807 18:12:43.942708 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 23742
Aug 07 18:12:43 dist-test-slave-gkw7 krb5kdc[23406](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126363, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.18.154.2@KRBTEST.COM
I20260807 18:12:43.960162 24146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:43.943814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37700 (local address 127.18.154.2:42845)
0807 18:12:43.944430 (+ 616us) server_negotiation.cc:206] Beginning negotiation
0807 18:12:43.944438 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:12:43.944467 (+ 29us) server_negotiation.cc:407] Connection header received
0807 18:12:43.944530 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:12:43.944536 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:12:43.944779 (+ 243us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:12:43.944918 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:12:43.946202 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:43.947265 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:43.948559 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:43.949238 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:43.949355 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:43.953707 (+ 4352us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:12:43.953722 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:12:43.953742 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:12:43.953783 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:12:43.957142 (+ 3359us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:12:43.957995 (+ 853us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:12:43.958003 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:12:43.958006 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:12:43.958143 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:12:43.958481 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:12:43.958485 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:12:43.958487 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:12:43.958944 (+ 457us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:12:43.959210 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0807 18:12:43.959489 (+ 279us) server_negotiation.cc:299] Negotiation successful
0807 18:12:43.959811 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":466,"thread_start_us":341,"threads_started":1}
W20260807 18:12:43.961671 23931 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:37700
W20260807 18:12:43.962888 19048 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 23742: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 18:12:43.963088 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 23742
I20260807 18:12:43.994499 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 23947
Aug 07 18:12:43 dist-test-slave-gkw7 krb5kdc[23406](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126363, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.18.154.3@KRBTEST.COM
I20260807 18:12:44.012013 24147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:43.995851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35958 (local address 127.18.154.3:39683)
0807 18:12:43.996579 (+ 728us) server_negotiation.cc:206] Beginning negotiation
0807 18:12:43.996587 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:12:43.996676 (+ 89us) server_negotiation.cc:407] Connection header received
0807 18:12:43.996834 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:12:43.996841 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:12:43.997056 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:12:43.997197 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:12:43.998600 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:43.999668 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:44.000941 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:44.001513 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:44.001711 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:44.006302 (+ 4591us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:12:44.006315 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:12:44.006332 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:12:44.006366 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:12:44.009260 (+ 2894us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:12:44.010018 (+ 758us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:12:44.010026 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:12:44.010028 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:12:44.010154 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:12:44.010483 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:12:44.010487 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:12:44.010489 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:12:44.010847 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:12:44.011090 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0807 18:12:44.011360 (+ 270us) server_negotiation.cc:299] Negotiation successful
0807 18:12:44.011630 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":573,"thread_start_us":400,"threads_started":1}
W20260807 18:12:44.013669 24122 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:35958
W20260807 18:12:44.015210 19048 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 23947: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 18:12:44.015386 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 23947
I20260807 18:12:44.039891 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 23425
I20260807 18:12:44.041208 23519 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:12:44.146416 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 23425
2026-08-07T18:12:44Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (4041 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126328808105-19048-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:12:44 dist-test-slave-gkw7 krb5kdc[24155](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:12:44 dist-test-slave-gkw7 krb5kdc[24155](info): set up 2 sockets
Aug 07 18:12:44 dist-test-slave-gkw7 krb5kdc[24155](info): commencing operation
krb5kdc: starting...
W20260807 18:12:45.517710 19048 mini_kdc.cc:121] Time spent starting KDC: real 1.323s user 0.003s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126328808105-19048-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126328808105-19048-0/krb5kdc/test-user.keytab.
Aug 07 18:12:45 dist-test-slave-gkw7 krb5kdc[24155](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126365, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:12:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:12:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.154.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.154.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.154.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.62.keytab.
Entry for principal kudu/127.18.154.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.62.keytab.
I20260807 18:12:45.754122 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126328808105-19048-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126328808105-19048-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126328808105-19048-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.154.62:40943
--webserver_interface=127.18.154.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.62.keytab
--principal=kudu/127.18.154.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.154.20:35729
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.154.62:40943
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126328808105-19048-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126328808105-19048-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126328808105-19048-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:12:46.089985 24174 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:12:46.090353 24174 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:12:46.090514 24174 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:12:46.097937 24174 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:12:46.098088 24174 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:12:46.098209 24174 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:12:46.098308 24174 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:12:46.098397 24174 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:12:46.108664 24174 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.154.20:35729
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126328808105-19048-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126328808105-19048-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.154.62:40943
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.62.keytab
--principal=kudu/127.18.154.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.154.62:40943
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.154.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126328808105-19048-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:12:46.110241 24174 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:12:46.111617 24174 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:12:46.124176 24180 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:12:46.124099 24188 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:12:46.124202 24182 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:12:46.125849 24174 server_base.cc:1034] running on GCE node
I20260807 18:12:46.126701 24174 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:12:46.128196 24174 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:12:46.129407 24174 hybrid_clock.cc:648] HybridClock initialized: now 1786126366129366 us; error 42 us; skew 500 ppm
I20260807 18:12:46.129814 24174 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:12:46 dist-test-slave-gkw7 krb5kdc[24155](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126366, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:12:46.138590 24174 init.cc:377] Logged in from keytab as kudu/127.18.154.62@KRBTEST.COM (short username kudu)
I20260807 18:12:46.140264 24174 webserver.cc:460] Webserver started at http://127.18.154.62:45477/ using document root <none> and password file <none>
I20260807 18:12:46.141033 24174 fs_manager.cc:359] Metadata directory not provided
I20260807 18:12:46.141189 24174 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:12:46.141578 24174 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:12:46.144755 24174 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126328808105-19048-0/minicluster-data/master-0/data/instance:
uuid: "8f3081cdf22c4614b322c6f5f6bbf23e"
format_stamp: "Formatted at 2026-08-07 18:12:46 on dist-test-slave-gkw7"
server_key: "b403de5a9e9e0fcb7b25ef6eb120f778"
server_key_iv: "40aedd54250006d7b67b648b836bcd66"
server_key_version: "encryptionkey@0"
I20260807 18:12:46.145687 24174 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126328808105-19048-0/minicluster-data/master-0/wal/instance:
uuid: "8f3081cdf22c4614b322c6f5f6bbf23e"
format_stamp: "Formatted at 2026-08-07 18:12:46 on dist-test-slave-gkw7"
server_key: "b403de5a9e9e0fcb7b25ef6eb120f778"
server_key_iv: "40aedd54250006d7b67b648b836bcd66"
server_key_version: "encryptionkey@0"
I20260807 18:12:46.151186 24174 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260807 18:12:46.154906 24203 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:12:46.156478 24174 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260807 18:12:46.156682 24174 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126328808105-19048-0/minicluster-data/master-0/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126328808105-19048-0/minicluster-data/master-0/wal
uuid: "8f3081cdf22c4614b322c6f5f6bbf23e"
format_stamp: "Formatted at 2026-08-07 18:12:46 on dist-test-slave-gkw7"
server_key: "b403de5a9e9e0fcb7b25ef6eb120f778"
server_key_iv: "40aedd54250006d7b67b648b836bcd66"
server_key_version: "encryptionkey@0"
I20260807 18:12:46.156891 24174 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126328808105-19048-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126328808105-19048-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126328808105-19048-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:12:46.177340 24174 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:12:46.181855 24174 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:12:46.182178 24174 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:12:46.201887 24174 rpc_server.cc:307] RPC server started. Bound to: 127.18.154.62:40943
I20260807 18:12:46.201910 24276 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.154.62:40943 every 8 connection(s)
I20260807 18:12:46.203358 24174 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
I20260807 18:12:46.207099 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 24174
I20260807 18:12:46.207356 19048 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126328808105-19048-0/minicluster-data/master-0/wal/instance
I20260807 18:12:46.207729 24278 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:12:46.208055 19048 external_mini_cluster.cc:1428] Setting key 9e29f470b4b425e1510fc5449b0add52
Aug 07 18:12:46 dist-test-slave-gkw7 krb5kdc[24155](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126365, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.154.62@KRBTEST.COM
I20260807 18:12:46.224109 24278 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f3081cdf22c4614b322c6f5f6bbf23e: Bootstrap starting.
I20260807 18:12:46.228824 24278 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8f3081cdf22c4614b322c6f5f6bbf23e: Neither blocks nor log segments found. Creating new log.
I20260807 18:12:46.230408 24278 log.cc:824] T 00000000000000000000000000000000 P 8f3081cdf22c4614b322c6f5f6bbf23e: Log is configured to *not* fsync() on all Append() calls
I20260807 18:12:46.234268 24278 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f3081cdf22c4614b322c6f5f6bbf23e: No bootstrap required, opened a new log
I20260807 18:12:46.234517 24282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:46.210113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38508 (local address 127.18.154.62:40943)
0807 18:12:46.211165 (+ 1052us) server_negotiation.cc:206] Beginning negotiation
0807 18:12:46.211188 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0807 18:12:46.211250 (+ 62us) server_negotiation.cc:407] Connection header received
0807 18:12:46.213444 (+ 2194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:12:46.213477 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:12:46.214083 (+ 606us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:12:46.214593 (+ 510us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:12:46.215893 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:46.218214 (+ 2321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:46.219517 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:46.220729 (+ 1212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:46.220934 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:46.225129 (+ 4195us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:12:46.225141 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:12:46.225161 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:12:46.225209 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:12:46.229664 (+ 4455us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:12:46.230393 (+ 729us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:12:46.230405 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:12:46.230414 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:12:46.230664 (+ 250us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:12:46.231081 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:12:46.231090 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:12:46.231097 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:12:46.231796 (+ 699us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:12:46.232121 (+ 325us) server_negotiation.cc:1035] Waiting for connection context
0807 18:12:46.232522 (+ 401us) server_negotiation.cc:299] Negotiation successful
0807 18:12:46.233002 (+ 480us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":760,"thread_start_us":438,"threads_started":1}
I20260807 18:12:46.243292 24278 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8f3081cdf22c4614b322c6f5f6bbf23e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f3081cdf22c4614b322c6f5f6bbf23e" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 40943 } }
I20260807 18:12:46.243693 24278 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8f3081cdf22c4614b322c6f5f6bbf23e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:12:46.243785 24278 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8f3081cdf22c4614b322c6f5f6bbf23e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f3081cdf22c4614b322c6f5f6bbf23e, State: Initialized, Role: FOLLOWER
I20260807 18:12:46.244340 24278 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8f3081cdf22c4614b322c6f5f6bbf23e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f3081cdf22c4614b322c6f5f6bbf23e" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 40943 } }
I20260807 18:12:46.244560 24278 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8f3081cdf22c4614b322c6f5f6bbf23e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:12:46.244665 24278 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8f3081cdf22c4614b322c6f5f6bbf23e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:12:46.244805 24278 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8f3081cdf22c4614b322c6f5f6bbf23e [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:12:46.247697 24278 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8f3081cdf22c4614b322c6f5f6bbf23e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f3081cdf22c4614b322c6f5f6bbf23e" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 40943 } }
I20260807 18:12:46.248230 24278 leader_election.cc:302] T 00000000000000000000000000000000 P 8f3081cdf22c4614b322c6f5f6bbf23e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f3081cdf22c4614b322c6f5f6bbf23e; no voters:
I20260807 18:12:46.248872 24278 leader_election.cc:288] T 00000000000000000000000000000000 P 8f3081cdf22c4614b322c6f5f6bbf23e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:12:46.249027 24288 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8f3081cdf22c4614b322c6f5f6bbf23e [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:12:46.249846 24288 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8f3081cdf22c4614b322c6f5f6bbf23e [term 1 LEADER]: Becoming Leader. State: Replica: 8f3081cdf22c4614b322c6f5f6bbf23e, State: Running, Role: LEADER
I20260807 18:12:46.250432 24288 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8f3081cdf22c4614b322c6f5f6bbf23e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f3081cdf22c4614b322c6f5f6bbf23e" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 40943 } }
I20260807 18:12:46.251259 24278 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8f3081cdf22c4614b322c6f5f6bbf23e [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:12:46.253516 24290 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f3081cdf22c4614b322c6f5f6bbf23e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8f3081cdf22c4614b322c6f5f6bbf23e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f3081cdf22c4614b322c6f5f6bbf23e" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 40943 } } }
I20260807 18:12:46.254093 24290 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f3081cdf22c4614b322c6f5f6bbf23e [sys.catalog]: This master's current role is: LEADER
I20260807 18:12:46.253988 24291 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f3081cdf22c4614b322c6f5f6bbf23e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8f3081cdf22c4614b322c6f5f6bbf23e. Latest consensus state: current_term: 1 leader_uuid: "8f3081cdf22c4614b322c6f5f6bbf23e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f3081cdf22c4614b322c6f5f6bbf23e" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 40943 } } }
I20260807 18:12:46.254292 24291 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f3081cdf22c4614b322c6f5f6bbf23e [sys.catalog]: This master's current role is: LEADER
I20260807 18:12:46.255836 24298 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:12:46.262135 24298 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:12:46.271857 24298 catalog_manager.cc:1347] Generated new cluster ID: 8f5f75a2a37b4860a726f99644be8728
I20260807 18:12:46.271981 24298 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:12:46.313822 24298 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:12:46.315248 24298 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:12:46.327795 24298 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8f3081cdf22c4614b322c6f5f6bbf23e: Generated new TSK 0
I20260807 18:12:46.329039 24298 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.154.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.154.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.154.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.1.keytab.
Entry for principal kudu/127.18.154.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.1.keytab.
I20260807 18:12:46.393757 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126328808105-19048-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126328808105-19048-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126328808105-19048-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126328808105-19048-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.154.1:0
--local_ip_for_outbound_sockets=127.18.154.1
--webserver_interface=127.18.154.1
--webserver_port=0
--tserver_master_addrs=127.18.154.62:40943
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.1.keytab
--principal=kudu/127.18.154.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.154.20:35729
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126328808105-19048-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126328808105-19048-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126328808105-19048-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:12:46.741191 24326 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:12:46.741500 24326 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:12:46.741688 24326 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:12:46.749032 24326 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:12:46.749151 24326 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:12:46.749265 24326 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.154.1
I20260807 18:12:46.759914 24326 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.154.20:35729
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126328808105-19048-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126328808105-19048-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.1.keytab
--principal=kudu/127.18.154.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.154.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126328808105-19048-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.154.1
--webserver_port=0
--tserver_master_addrs=127.18.154.62:40943
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.154.1
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126328808105-19048-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:12:46.761372 24326 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:12:46.762843 24326 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:12:46.776360 24332 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:12:46.776321 24335 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:12:46.776512 24333 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:12:46.777606 24326 server_base.cc:1034] running on GCE node
I20260807 18:12:46.778195 24326 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:12:46.779168 24326 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:12:46.780367 24326 hybrid_clock.cc:648] HybridClock initialized: now 1786126366780325 us; error 35 us; skew 500 ppm
I20260807 18:12:46.780709 24326 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:12:46 dist-test-slave-gkw7 krb5kdc[24155](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126366, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:12:46.784077 24326 init.cc:377] Logged in from keytab as kudu/127.18.154.1@KRBTEST.COM (short username kudu)
I20260807 18:12:46.785702 24326 webserver.cc:460] Webserver started at http://127.18.154.1:39309/ using document root <none> and password file <none>
I20260807 18:12:46.786491 24326 fs_manager.cc:359] Metadata directory not provided
I20260807 18:12:46.786652 24326 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:12:46.787005 24326 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:12:46.790040 24326 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126328808105-19048-0/minicluster-data/ts-0/data/instance:
uuid: "139aac56201b403e80073c29a820d912"
format_stamp: "Formatted at 2026-08-07 18:12:46 on dist-test-slave-gkw7"
server_key: "36b7db7d0c32f16c0cd6821ac9fe96c9"
server_key_iv: "71bacf1358ca49f00686c084f9620fc8"
server_key_version: "encryptionkey@0"
I20260807 18:12:46.790853 24326 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126328808105-19048-0/minicluster-data/ts-0/wal/instance:
uuid: "139aac56201b403e80073c29a820d912"
format_stamp: "Formatted at 2026-08-07 18:12:46 on dist-test-slave-gkw7"
server_key: "36b7db7d0c32f16c0cd6821ac9fe96c9"
server_key_iv: "71bacf1358ca49f00686c084f9620fc8"
server_key_version: "encryptionkey@0"
I20260807 18:12:46.796474 24326 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 18:12:46.800252 24342 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:12:46.801785 24326 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:12:46.801983 24326 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126328808105-19048-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126328808105-19048-0/minicluster-data/ts-0/wal
uuid: "139aac56201b403e80073c29a820d912"
format_stamp: "Formatted at 2026-08-07 18:12:46 on dist-test-slave-gkw7"
server_key: "36b7db7d0c32f16c0cd6821ac9fe96c9"
server_key_iv: "71bacf1358ca49f00686c084f9620fc8"
server_key_version: "encryptionkey@0"
I20260807 18:12:46.802207 24326 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126328808105-19048-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126328808105-19048-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126328808105-19048-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:12:46.833881 24326 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:12:46.838513 24326 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:12:46.838832 24326 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:12:46.839978 24326 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:12:46.841387 24326 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:12:46.841498 24326 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:12:46.841676 24326 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:12:46.841787 24326 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:12:46.877288 24326 rpc_server.cc:307] RPC server started. Bound to: 127.18.154.1:45479
I20260807 18:12:46.877348 24505 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.154.1:45479 every 8 connection(s)
I20260807 18:12:46.878705 24326 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126328808105-19048-0/minicluster-data/ts-0/data/info.pb
I20260807 18:12:46.888411 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 24326
I20260807 18:12:46.888640 19048 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126328808105-19048-0/minicluster-data/ts-0/wal/instance
I20260807 18:12:46.889500 19048 external_mini_cluster.cc:1428] Setting key 1c9df1572618db4626fca830e3d4bce3
Aug 07 18:12:46 dist-test-slave-gkw7 krb5kdc[24155](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126366, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.1@KRBTEST.COM for kudu/127.18.154.62@KRBTEST.COM
I20260807 18:12:46.900691 24509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:46.881423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.1:50103 (local address 127.18.154.62:40943)
0807 18:12:46.881920 (+ 497us) server_negotiation.cc:206] Beginning negotiation
0807 18:12:46.881924 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:12:46.882952 (+ 1028us) server_negotiation.cc:407] Connection header received
0807 18:12:46.884822 (+ 1870us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:12:46.884828 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:12:46.884922 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:12:46.885048 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:12:46.886646 (+ 1598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:46.887403 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:46.889879 (+ 2476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:46.890446 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:46.890565 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:46.894753 (+ 4188us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:12:46.894758 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:12:46.894762 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:12:46.894794 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:12:46.897420 (+ 2626us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:12:46.898257 (+ 837us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:12:46.898261 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:12:46.898263 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:12:46.898386 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:12:46.898866 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:12:46.898870 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:12:46.898872 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:12:46.899143 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:12:46.899306 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0807 18:12:46.900070 (+ 764us) server_negotiation.cc:299] Negotiation successful
0807 18:12:46.900318 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":388,"thread_start_us":256,"threads_started":1}
I20260807 18:12:46.901844 24510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:46.882075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:40943 (local address 127.18.154.1:50103)
0807 18:12:46.882783 (+ 708us) negotiation.cc:107] Waiting for socket to connect
0807 18:12:46.882830 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 18:12:46.884541 (+ 1711us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:12:46.885298 (+ 757us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:12:46.885306 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:12:46.885892 (+ 586us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:12:46.886473 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:12:46.886491 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:12:46.887582 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:12:46.887586 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:12:46.889711 (+ 2125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:12:46.889731 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:12:46.890691 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:12:46.890701 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:12:46.890940 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:46.891709 (+ 769us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:12:46.891741 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:12:46.894503 (+ 2762us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:12:46.897605 (+ 3102us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:12:46.897615 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:12:46.897638 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:12:46.898095 (+ 457us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:12:46.898505 (+ 410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:12:46.898509 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:12:46.898513 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:12:46.898733 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:12:46.899291 (+ 558us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:12:46.899300 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:12:46.899669 (+ 369us) client_negotiation.cc:769] Sending connection context
0807 18:12:46.899959 (+ 290us) client_negotiation.cc:240] Negotiation successful
0807 18:12:46.900524 (+ 565us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":461,"thread_start_us":284,"threads_started":1}
I20260807 18:12:46.903676 24507 heartbeater.cc:347] Connected to a master server at 127.18.154.62:40943
I20260807 18:12:46.904018 24507 heartbeater.cc:464] Registering TS with master...
I20260807 18:12:46.904774 24507 heartbeater.cc:511] Master 127.18.154.62:40943 requested a full tablet report, sending...
I20260807 18:12:46.907317 24224 ts_manager.cc:194] Registered new tserver with Master: 139aac56201b403e80073c29a820d912 (127.18.154.1:45479)
I20260807 18:12:46.909606 24224 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.154.1@KRBTEST.COM'} at 127.18.154.1:50103
WARNING: no policy specified for kudu/127.18.154.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.154.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.154.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.2.keytab.
Entry for principal kudu/127.18.154.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.2.keytab.
I20260807 18:12:46.941493 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126328808105-19048-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126328808105-19048-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126328808105-19048-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126328808105-19048-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.154.2:0
--local_ip_for_outbound_sockets=127.18.154.2
--webserver_interface=127.18.154.2
--webserver_port=0
--tserver_master_addrs=127.18.154.62:40943
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.2.keytab
--principal=kudu/127.18.154.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.154.20:35729
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126328808105-19048-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126328808105-19048-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126328808105-19048-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:12:47.303500 24517 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:12:47.303812 24517 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:12:47.304000 24517 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:12:47.311607 24517 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:12:47.311735 24517 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:12:47.311949 24517 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.154.2
I20260807 18:12:47.322160 24517 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.154.20:35729
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126328808105-19048-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126328808105-19048-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.2.keytab
--principal=kudu/127.18.154.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.154.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126328808105-19048-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.154.2
--webserver_port=0
--tserver_master_addrs=127.18.154.62:40943
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.154.2
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126328808105-19048-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:12:47.323587 24517 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:12:47.325009 24517 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:12:47.338778 24523 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:12:47.338989 24525 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:12:47.338878 24522 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:12:47.340131 24517 server_base.cc:1034] running on GCE node
I20260807 18:12:47.340934 24517 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:12:47.342159 24517 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:12:47.343400 24517 hybrid_clock.cc:648] HybridClock initialized: now 1786126367343360 us; error 58 us; skew 500 ppm
I20260807 18:12:47.343735 24517 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:12:47 dist-test-slave-gkw7 krb5kdc[24155](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126367, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:12:47.347093 24517 init.cc:377] Logged in from keytab as kudu/127.18.154.2@KRBTEST.COM (short username kudu)
I20260807 18:12:47.348719 24517 webserver.cc:460] Webserver started at http://127.18.154.2:34215/ using document root <none> and password file <none>
I20260807 18:12:47.349466 24517 fs_manager.cc:359] Metadata directory not provided
I20260807 18:12:47.349664 24517 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:12:47.350008 24517 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:12:47.353192 24517 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126328808105-19048-0/minicluster-data/ts-1/data/instance:
uuid: "1d1b9b75ef3642659514c40e73363e36"
format_stamp: "Formatted at 2026-08-07 18:12:47 on dist-test-slave-gkw7"
server_key: "447d00e6dc8c27a5f9b9c53b36913979"
server_key_iv: "0cd38a0bedc5565afb736edde57b38e5"
server_key_version: "encryptionkey@0"
I20260807 18:12:47.354255 24517 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126328808105-19048-0/minicluster-data/ts-1/wal/instance:
uuid: "1d1b9b75ef3642659514c40e73363e36"
format_stamp: "Formatted at 2026-08-07 18:12:47 on dist-test-slave-gkw7"
server_key: "447d00e6dc8c27a5f9b9c53b36913979"
server_key_iv: "0cd38a0bedc5565afb736edde57b38e5"
server_key_version: "encryptionkey@0"
I20260807 18:12:47.360172 24517 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 18:12:47.363902 24535 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:12:47.365423 24517 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 18:12:47.365669 24517 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126328808105-19048-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126328808105-19048-0/minicluster-data/ts-1/wal
uuid: "1d1b9b75ef3642659514c40e73363e36"
format_stamp: "Formatted at 2026-08-07 18:12:47 on dist-test-slave-gkw7"
server_key: "447d00e6dc8c27a5f9b9c53b36913979"
server_key_iv: "0cd38a0bedc5565afb736edde57b38e5"
server_key_version: "encryptionkey@0"
I20260807 18:12:47.365903 24517 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126328808105-19048-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126328808105-19048-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126328808105-19048-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:12:47.391341 24517 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:12:47.396003 24517 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:12:47.396327 24517 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:12:47.397400 24517 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:12:47.398869 24517 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:12:47.398978 24517 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:12:47.399147 24517 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:12:47.399225 24517 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:12:47.434535 24517 rpc_server.cc:307] RPC server started. Bound to: 127.18.154.2:35799
I20260807 18:12:47.434581 24697 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.154.2:35799 every 8 connection(s)
I20260807 18:12:47.435954 24517 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126328808105-19048-0/minicluster-data/ts-1/data/info.pb
I20260807 18:12:47.446363 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 24517
I20260807 18:12:47.446558 19048 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126328808105-19048-0/minicluster-data/ts-1/wal/instance
I20260807 18:12:47.447142 19048 external_mini_cluster.cc:1428] Setting key 6e572accf6a60d8fd393ef111cbb1353
Aug 07 18:12:47 dist-test-slave-gkw7 krb5kdc[24155](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126367, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.2@KRBTEST.COM for kudu/127.18.154.62@KRBTEST.COM
I20260807 18:12:47.458539 24701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:47.438802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.2:36689 (local address 127.18.154.62:40943)
0807 18:12:47.439275 (+ 473us) server_negotiation.cc:206] Beginning negotiation
0807 18:12:47.439280 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:12:47.440234 (+ 954us) server_negotiation.cc:407] Connection header received
0807 18:12:47.442437 (+ 2203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:12:47.442443 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:12:47.442541 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:12:47.442666 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:12:47.444279 (+ 1613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:47.445009 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:47.447190 (+ 2181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:47.447765 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:47.447879 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:47.452182 (+ 4303us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:12:47.452192 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:12:47.452198 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:12:47.452247 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:12:47.455129 (+ 2882us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:12:47.455948 (+ 819us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:12:47.455952 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:12:47.455954 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:12:47.456096 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:12:47.456575 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:12:47.456587 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:12:47.456590 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:12:47.456883 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:12:47.457060 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0807 18:12:47.457914 (+ 854us) server_negotiation.cc:299] Negotiation successful
0807 18:12:47.458156 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":363,"thread_start_us":252,"threads_started":1}
I20260807 18:12:47.459672 24702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:47.439315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:40943 (local address 127.18.154.2:36689)
0807 18:12:47.440059 (+ 744us) negotiation.cc:107] Waiting for socket to connect
0807 18:12:47.440099 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 18:12:47.442137 (+ 2038us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:12:47.442932 (+ 795us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:12:47.442940 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:12:47.443494 (+ 554us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:12:47.444090 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:12:47.444108 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:12:47.445180 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:12:47.445184 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:12:47.447015 (+ 1831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:12:47.447038 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:12:47.448027 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:12:47.448042 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:12:47.448296 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:47.449067 (+ 771us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:12:47.449124 (+ 57us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:12:47.451860 (+ 2736us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:12:47.455294 (+ 3434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:12:47.455304 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:12:47.455323 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:12:47.455785 (+ 462us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:12:47.456226 (+ 441us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:12:47.456231 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:12:47.456235 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:12:47.456444 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:12:47.457023 (+ 579us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:12:47.457032 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:12:47.457421 (+ 389us) client_negotiation.cc:769] Sending connection context
0807 18:12:47.457783 (+ 362us) client_negotiation.cc:240] Negotiation successful
0807 18:12:47.458387 (+ 604us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":507,"thread_start_us":304,"threads_started":1}
I20260807 18:12:47.461266 24698 heartbeater.cc:347] Connected to a master server at 127.18.154.62:40943
I20260807 18:12:47.461679 24698 heartbeater.cc:464] Registering TS with master...
I20260807 18:12:47.462514 24698 heartbeater.cc:511] Master 127.18.154.62:40943 requested a full tablet report, sending...
I20260807 18:12:47.464150 24224 ts_manager.cc:194] Registered new tserver with Master: 1d1b9b75ef3642659514c40e73363e36 (127.18.154.2:35799)
I20260807 18:12:47.465503 24224 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.154.2@KRBTEST.COM'} at 127.18.154.2:36689
WARNING: no policy specified for kudu/127.18.154.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.154.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.154.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.3.keytab.
Entry for principal kudu/127.18.154.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.3.keytab.
I20260807 18:12:47.501766 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126328808105-19048-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126328808105-19048-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126328808105-19048-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126328808105-19048-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.154.3:0
--local_ip_for_outbound_sockets=127.18.154.3
--webserver_interface=127.18.154.3
--webserver_port=0
--tserver_master_addrs=127.18.154.62:40943
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.3.keytab
--principal=kudu/127.18.154.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.154.20:35729
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126328808105-19048-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126328808105-19048-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126328808105-19048-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:12:47.844117 24708 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:12:47.844413 24708 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:12:47.844514 24708 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:12:47.851838 24708 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:12:47.851951 24708 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:12:47.852082 24708 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.154.3
I20260807 18:12:47.862591 24708 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.154.20:35729
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126328808105-19048-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126328808105-19048-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.3.keytab
--principal=kudu/127.18.154.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.154.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126328808105-19048-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.154.3
--webserver_port=0
--tserver_master_addrs=127.18.154.62:40943
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.154.3
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126328808105-19048-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:12:47.864050 24708 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:12:47.865391 24708 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:12:47.879439 24716 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:12:47.879433 24714 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:12:47.880172 24721 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:12:47.880429 24708 server_base.cc:1034] running on GCE node
I20260807 18:12:47.881160 24708 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:12:47.882287 24708 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:12:47.883565 24708 hybrid_clock.cc:648] HybridClock initialized: now 1786126367883527 us; error 31 us; skew 500 ppm
I20260807 18:12:47.883931 24708 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:12:47 dist-test-slave-gkw7 krb5kdc[24155](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126367, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:12:47.887403 24708 init.cc:377] Logged in from keytab as kudu/127.18.154.3@KRBTEST.COM (short username kudu)
I20260807 18:12:47.889106 24708 webserver.cc:460] Webserver started at http://127.18.154.3:42187/ using document root <none> and password file <none>
I20260807 18:12:47.889964 24708 fs_manager.cc:359] Metadata directory not provided
I20260807 18:12:47.890121 24708 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:12:47.890468 24708 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:12:47.893766 24708 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126328808105-19048-0/minicluster-data/ts-2/data/instance:
uuid: "bf7cefaf3b554a23a4e6d2f1df7800f1"
format_stamp: "Formatted at 2026-08-07 18:12:47 on dist-test-slave-gkw7"
server_key: "8e37803e0fa3dcf13c343541450871d8"
server_key_iv: "9167c842525a6933fde25152c185c8a8"
server_key_version: "encryptionkey@0"
I20260807 18:12:47.894625 24708 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126328808105-19048-0/minicluster-data/ts-2/wal/instance:
uuid: "bf7cefaf3b554a23a4e6d2f1df7800f1"
format_stamp: "Formatted at 2026-08-07 18:12:47 on dist-test-slave-gkw7"
server_key: "8e37803e0fa3dcf13c343541450871d8"
server_key_iv: "9167c842525a6933fde25152c185c8a8"
server_key_version: "encryptionkey@0"
I20260807 18:12:47.900373 24708 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 18:12:47.904456 24733 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:12:47.906180 24708 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:12:47.906400 24708 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126328808105-19048-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126328808105-19048-0/minicluster-data/ts-2/wal
uuid: "bf7cefaf3b554a23a4e6d2f1df7800f1"
format_stamp: "Formatted at 2026-08-07 18:12:47 on dist-test-slave-gkw7"
server_key: "8e37803e0fa3dcf13c343541450871d8"
server_key_iv: "9167c842525a6933fde25152c185c8a8"
server_key_version: "encryptionkey@0"
I20260807 18:12:47.906651 24708 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126328808105-19048-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126328808105-19048-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126328808105-19048-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:12:47.912781 24507 heartbeater.cc:503] Master 127.18.154.62:40943 was elected leader, sending a full tablet report...
I20260807 18:12:47.920615 24708 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:12:47.925522 24708 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:12:47.925930 24708 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:12:47.927006 24708 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:12:47.928572 24708 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:12:47.928653 24708 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:12:47.928831 24708 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:12:47.928906 24708 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:12:47.965796 24708 rpc_server.cc:307] RPC server started. Bound to: 127.18.154.3:42471
I20260807 18:12:47.965839 24884 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.154.3:42471 every 8 connection(s)
I20260807 18:12:47.967319 24708 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126328808105-19048-0/minicluster-data/ts-2/data/info.pb
I20260807 18:12:47.975944 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 24708
I20260807 18:12:47.976143 19048 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786126328808105-19048-0/minicluster-data/ts-2/wal/instance
I20260807 18:12:47.976703 19048 external_mini_cluster.cc:1428] Setting key a41daa142589f6db161e1f6b6f225bf2
Aug 07 18:12:47 dist-test-slave-gkw7 krb5kdc[24155](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126367, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.3@KRBTEST.COM for kudu/127.18.154.62@KRBTEST.COM
I20260807 18:12:47.989863 24890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:47.970032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.3:58339 (local address 127.18.154.62:40943)
0807 18:12:47.970561 (+ 529us) server_negotiation.cc:206] Beginning negotiation
0807 18:12:47.970566 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:12:47.971639 (+ 1073us) server_negotiation.cc:407] Connection header received
0807 18:12:47.973869 (+ 2230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:12:47.973874 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:12:47.973957 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:12:47.974061 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:12:47.975732 (+ 1671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:47.976997 (+ 1265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:47.979814 (+ 2817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:47.980349 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:47.980453 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:47.984105 (+ 3652us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:12:47.984110 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:12:47.984114 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:12:47.984153 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:12:47.987040 (+ 2887us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:12:47.987788 (+ 748us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:12:47.987792 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:12:47.987794 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:12:47.987909 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:12:47.988308 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:12:47.988311 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:12:47.988313 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:12:47.988569 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:12:47.988692 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0807 18:12:47.989342 (+ 650us) server_negotiation.cc:299] Negotiation successful
0807 18:12:47.989504 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":380,"thread_start_us":239,"threads_started":1}
I20260807 18:12:47.990996 24892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:47.970653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:40943 (local address 127.18.154.3:58339)
0807 18:12:47.971444 (+ 791us) negotiation.cc:107] Waiting for socket to connect
0807 18:12:47.971485 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 18:12:47.973520 (+ 2035us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:12:47.974298 (+ 778us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:12:47.974307 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:12:47.974881 (+ 574us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:12:47.975513 (+ 632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:12:47.975532 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:12:47.977211 (+ 1679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:12:47.977219 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:12:47.979659 (+ 2440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:12:47.979674 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:12:47.980582 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:12:47.980587 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:12:47.980841 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:47.981507 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:12:47.981598 (+ 91us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:12:47.983888 (+ 2290us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:12:47.987219 (+ 3331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:12:47.987228 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:12:47.987242 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:12:47.987659 (+ 417us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:12:47.988001 (+ 342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:12:47.988004 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:12:47.988006 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:12:47.988203 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:12:47.988674 (+ 471us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:12:47.988680 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:12:47.989015 (+ 335us) client_negotiation.cc:769] Sending connection context
0807 18:12:47.989243 (+ 228us) client_negotiation.cc:240] Negotiation successful
0807 18:12:47.989703 (+ 460us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":525,"thread_start_us":277,"threads_started":1}
I20260807 18:12:47.992555 24887 heartbeater.cc:347] Connected to a master server at 127.18.154.62:40943
I20260807 18:12:47.992910 24887 heartbeater.cc:464] Registering TS with master...
I20260807 18:12:47.993777 24887 heartbeater.cc:511] Master 127.18.154.62:40943 requested a full tablet report, sending...
I20260807 18:12:47.995334 24223 ts_manager.cc:194] Registered new tserver with Master: bf7cefaf3b554a23a4e6d2f1df7800f1 (127.18.154.3:42471)
I20260807 18:12:47.996227 24223 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.154.3@KRBTEST.COM'} at 127.18.154.3:58339
I20260807 18:12:48.004983 19048 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 18:12:48 dist-test-slave-gkw7 krb5kdc[24155](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126368, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 18:12:48 dist-test-slave-gkw7 krb5kdc[24155](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126368, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.154.1@KRBTEST.COM
I20260807 18:12:48.063990 24899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:48.048126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53968 (local address 127.18.154.1:45479)
0807 18:12:48.048786 (+ 660us) server_negotiation.cc:206] Beginning negotiation
0807 18:12:48.048793 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:12:48.048822 (+ 29us) server_negotiation.cc:407] Connection header received
0807 18:12:48.048886 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:12:48.048892 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:12:48.049122 (+ 230us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:12:48.049313 (+ 191us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:12:48.050836 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:48.052047 (+ 1211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:48.053391 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:48.054036 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:48.054147 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:48.058126 (+ 3979us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:12:48.058167 (+ 41us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:12:48.058183 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:12:48.058220 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:12:48.061248 (+ 3028us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:12:48.062025 (+ 777us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:12:48.062032 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:12:48.062034 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:12:48.062194 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:12:48.062543 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:12:48.062546 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:12:48.062548 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:12:48.062900 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:12:48.063133 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0807 18:12:48.063343 (+ 210us) server_negotiation.cc:299] Negotiation successful
0807 18:12:48.063551 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":501,"thread_start_us":349,"threads_started":1}
Aug 07 18:12:48 dist-test-slave-gkw7 krb5kdc[24155](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126368, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.154.62@KRBTEST.COM
I20260807 18:12:48.082914 24890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:48.068917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38518 (local address 127.18.154.62:40943)
0807 18:12:48.069180 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0807 18:12:48.069188 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:12:48.069349 (+ 161us) server_negotiation.cc:407] Connection header received
0807 18:12:48.069606 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:12:48.069610 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:12:48.069683 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:12:48.069794 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:12:48.071201 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:48.071960 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:48.073225 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:48.073717 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:48.073868 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:48.077853 (+ 3985us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:12:48.077858 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:12:48.077861 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:12:48.077898 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:12:48.080425 (+ 2527us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:12:48.081030 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:12:48.081033 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:12:48.081036 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:12:48.081185 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:12:48.081503 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:12:48.081506 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:12:48.081508 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:12:48.081907 (+ 399us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:12:48.082089 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0807 18:12:48.082377 (+ 288us) server_negotiation.cc:299] Negotiation successful
0807 18:12:48.082548 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
W20260807 18:12:48.083523 24223 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38518
I20260807 18:12:48.084604 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 24326
I20260807 18:12:48.097684 24899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:48.085512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53982 (local address 127.18.154.1:45479)
0807 18:12:48.085760 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0807 18:12:48.085767 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:12:48.086015 (+ 248us) server_negotiation.cc:407] Connection header received
0807 18:12:48.086205 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:12:48.086234 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:12:48.086334 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:12:48.086446 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:12:48.087711 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:48.088412 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:48.089707 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:48.090121 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:48.090286 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:48.092540 (+ 2254us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:12:48.092545 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:12:48.092548 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:12:48.092580 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:12:48.095075 (+ 2495us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:12:48.095708 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:12:48.095714 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:12:48.095718 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:12:48.095853 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:12:48.096244 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:12:48.096249 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:12:48.096253 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:12:48.096510 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:12:48.096770 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0807 18:12:48.097088 (+ 318us) server_negotiation.cc:299] Negotiation successful
0807 18:12:48.097334 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260807 18:12:48.098290 24499 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:12:48.182222 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 24326
I20260807 18:12:48.206904 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 24517
Aug 07 18:12:48 dist-test-slave-gkw7 krb5kdc[24155](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126368, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.154.2@KRBTEST.COM
I20260807 18:12:48.223769 24908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:48.207924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54126 (local address 127.18.154.2:35799)
0807 18:12:48.208509 (+ 585us) server_negotiation.cc:206] Beginning negotiation
0807 18:12:48.208517 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:12:48.208544 (+ 27us) server_negotiation.cc:407] Connection header received
0807 18:12:48.208607 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:12:48.208612 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:12:48.208853 (+ 241us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:12:48.208985 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:12:48.210518 (+ 1533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:48.211690 (+ 1172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:48.212960 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:48.213513 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:48.213743 (+ 230us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:48.217844 (+ 4101us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:12:48.217857 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:12:48.217872 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:12:48.217908 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:12:48.220744 (+ 2836us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:12:48.221402 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:12:48.221410 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:12:48.221413 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:12:48.221746 (+ 333us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:12:48.222130 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:12:48.222135 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:12:48.222137 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:12:48.222563 (+ 426us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:12:48.222829 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0807 18:12:48.223106 (+ 277us) server_negotiation.cc:299] Negotiation successful
0807 18:12:48.223392 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":444,"thread_start_us":298,"threads_started":1}
I20260807 18:12:48.225195 24689 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:12:48.309891 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 24517
I20260807 18:12:48.332726 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 24708
Aug 07 18:12:48 dist-test-slave-gkw7 krb5kdc[24155](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126368, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.154.3@KRBTEST.COM
I20260807 18:12:48.350065 24911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:48.333829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46924 (local address 127.18.154.3:42471)
0807 18:12:48.334500 (+ 671us) server_negotiation.cc:206] Beginning negotiation
0807 18:12:48.334508 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:12:48.334538 (+ 30us) server_negotiation.cc:407] Connection header received
0807 18:12:48.334600 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:12:48.334606 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:12:48.334830 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:12:48.334962 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:12:48.336291 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:48.337333 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:48.338749 (+ 1416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:48.339323 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:48.339424 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:48.343930 (+ 4506us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:12:48.343943 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:12:48.343959 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:12:48.343995 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:12:48.346953 (+ 2958us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:12:48.347972 (+ 1019us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:12:48.347982 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:12:48.347987 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:12:48.348155 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:12:48.348540 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:12:48.348543 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:12:48.348545 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:12:48.348898 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:12:48.349180 (+ 282us) server_negotiation.cc:1035] Waiting for connection context
0807 18:12:48.349359 (+ 179us) server_negotiation.cc:299] Negotiation successful
0807 18:12:48.349657 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":519,"thread_start_us":328,"threads_started":1}
I20260807 18:12:48.351485 24877 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:12:48.434360 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 24708
I20260807 18:12:48.456664 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 24174
I20260807 18:12:48.458117 24272 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:12:48.563664 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 24174
2026-08-07T18:12:48Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (4419 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126328808105-19048-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:12:48 dist-test-slave-gkw7 krb5kdc[24916](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:12:48 dist-test-slave-gkw7 krb5kdc[24916](info): set up 2 sockets
Aug 07 18:12:48 dist-test-slave-gkw7 krb5kdc[24916](info): commencing operation
krb5kdc: starting...
W20260807 18:12:49.849960 19048 mini_kdc.cc:121] Time spent starting KDC: real 1.236s user 0.000s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126328808105-19048-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126328808105-19048-0/krb5kdc/test-user.keytab.
Aug 07 18:12:49 dist-test-slave-gkw7 krb5kdc[24916](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126369, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:12:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:12:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.154.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.154.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.154.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.62.keytab.
Entry for principal kudu/127.18.154.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.62.keytab.
I20260807 18:12:50.085376 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126328808105-19048-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126328808105-19048-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126328808105-19048-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.154.62:45139
--webserver_interface=127.18.154.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.62.keytab
--principal=kudu/127.18.154.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.154.20:38131
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.154.62:45139
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126328808105-19048-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126328808105-19048-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126328808105-19048-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:12:50.424252 24935 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:12:50.424619 24935 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:12:50.424784 24935 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:12:50.432268 24935 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:12:50.432425 24935 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:12:50.432533 24935 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:12:50.432637 24935 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:12:50.432729 24935 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:12:50.443177 24935 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.154.20:38131
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126328808105-19048-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126328808105-19048-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.154.62:45139
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.62.keytab
--principal=kudu/127.18.154.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.154.62:45139
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.154.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126328808105-19048-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:12:50.444638 24935 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:12:50.446081 24935 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:12:50.458561 24944 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:12:50.458588 24942 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:12:50.459278 24941 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:12:50.459550 24935 server_base.cc:1034] running on GCE node
I20260807 18:12:50.460386 24935 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:12:50.461807 24935 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:12:50.463029 24935 hybrid_clock.cc:648] HybridClock initialized: now 1786126370462991 us; error 25 us; skew 500 ppm
I20260807 18:12:50.463374 24935 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:12:50 dist-test-slave-gkw7 krb5kdc[24916](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126370, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:12:50.466543 24935 init.cc:377] Logged in from keytab as kudu/127.18.154.62@KRBTEST.COM (short username kudu)
I20260807 18:12:50.467751 24935 fs_manager.cc:359] Metadata directory not provided
I20260807 18:12:50.467841 24935 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:12:50.468117 24935 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:12:50.471225 24935 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126328808105-19048-0/minicluster-data/master-0/data/instance:
uuid: "d6ea032973024e708c5883eefe5d3552"
format_stamp: "Formatted at 2026-08-07 18:12:50 on dist-test-slave-gkw7"
server_key: "f01ee95fc3f00444f2136fef3a9c7453"
server_key_iv: "b5ce152b016efb1c1622dd3104890b0b"
server_key_version: "encryptionkey@0"
I20260807 18:12:50.471993 24935 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126328808105-19048-0/minicluster-data/master-0/wal/instance:
uuid: "d6ea032973024e708c5883eefe5d3552"
format_stamp: "Formatted at 2026-08-07 18:12:50 on dist-test-slave-gkw7"
server_key: "f01ee95fc3f00444f2136fef3a9c7453"
server_key_iv: "b5ce152b016efb1c1622dd3104890b0b"
server_key_version: "encryptionkey@0"
I20260807 18:12:50.477242 24935 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 18:12:50.480988 24953 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:12:50.482467 24935 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:12:50.482678 24935 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126328808105-19048-0/minicluster-data/master-0/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126328808105-19048-0/minicluster-data/master-0/wal
uuid: "d6ea032973024e708c5883eefe5d3552"
format_stamp: "Formatted at 2026-08-07 18:12:50 on dist-test-slave-gkw7"
server_key: "f01ee95fc3f00444f2136fef3a9c7453"
server_key_iv: "b5ce152b016efb1c1622dd3104890b0b"
server_key_version: "encryptionkey@0"
I20260807 18:12:50.482913 24935 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126328808105-19048-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126328808105-19048-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126328808105-19048-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:12:50.501998 24935 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:12:50.506493 24935 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:12:50.506810 24935 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:12:50.525854 24935 rpc_server.cc:307] RPC server started. Bound to: 127.18.154.62:45139
I20260807 18:12:50.525871 25021 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.154.62:45139 every 8 connection(s)
I20260807 18:12:50.526834 24935 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
I20260807 18:12:50.528594 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 24935
I20260807 18:12:50.528824 19048 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126328808105-19048-0/minicluster-data/master-0/wal/instance
I20260807 18:12:50.529681 19048 external_mini_cluster.cc:1428] Setting key da34c375e9da2e6ed83945c510b65e79
I20260807 18:12:50.531880 25022 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 07 18:12:50 dist-test-slave-gkw7 krb5kdc[24916](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126369, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.154.62@KRBTEST.COM
I20260807 18:12:50.545919 25022 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d6ea032973024e708c5883eefe5d3552: Bootstrap starting.
I20260807 18:12:50.550318 25022 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d6ea032973024e708c5883eefe5d3552: Neither blocks nor log segments found. Creating new log.
I20260807 18:12:50.551614 25022 log.cc:824] T 00000000000000000000000000000000 P d6ea032973024e708c5883eefe5d3552: Log is configured to *not* fsync() on all Append() calls
I20260807 18:12:50.555055 25022 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d6ea032973024e708c5883eefe5d3552: No bootstrap required, opened a new log
I20260807 18:12:50.555274 25027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:50.531894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33816 (local address 127.18.154.62:45139)
0807 18:12:50.532805 (+ 911us) server_negotiation.cc:206] Beginning negotiation
0807 18:12:50.532817 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 18:12:50.532858 (+ 41us) server_negotiation.cc:407] Connection header received
0807 18:12:50.534542 (+ 1684us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:12:50.534566 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:12:50.534972 (+ 406us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:12:50.535326 (+ 354us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:12:50.537245 (+ 1919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:50.538766 (+ 1521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:50.540506 (+ 1740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:50.541209 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:50.541356 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:50.546427 (+ 5071us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:12:50.546439 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:12:50.546460 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:12:50.546525 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:12:50.550620 (+ 4095us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:12:50.551424 (+ 804us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:12:50.551434 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:12:50.551443 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:12:50.551667 (+ 224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:12:50.552105 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:12:50.552114 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:12:50.552120 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:12:50.552742 (+ 622us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:12:50.553052 (+ 310us) server_negotiation.cc:1035] Waiting for connection context
0807 18:12:50.553460 (+ 408us) server_negotiation.cc:299] Negotiation successful
0807 18:12:50.553945 (+ 485us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":688,"thread_start_us":406,"threads_started":1}
I20260807 18:12:50.562541 25022 raft_consensus.cc:348] T 00000000000000000000000000000000 P d6ea032973024e708c5883eefe5d3552 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6ea032973024e708c5883eefe5d3552" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 45139 } }
I20260807 18:12:50.562880 25022 raft_consensus.cc:374] T 00000000000000000000000000000000 P d6ea032973024e708c5883eefe5d3552 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:12:50.562953 25022 raft_consensus.cc:729] T 00000000000000000000000000000000 P d6ea032973024e708c5883eefe5d3552 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6ea032973024e708c5883eefe5d3552, State: Initialized, Role: FOLLOWER
I20260807 18:12:50.563478 25022 consensus_queue.cc:260] T 00000000000000000000000000000000 P d6ea032973024e708c5883eefe5d3552 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6ea032973024e708c5883eefe5d3552" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 45139 } }
I20260807 18:12:50.563681 25022 raft_consensus.cc:388] T 00000000000000000000000000000000 P d6ea032973024e708c5883eefe5d3552 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:12:50.563859 25022 raft_consensus.cc:482] T 00000000000000000000000000000000 P d6ea032973024e708c5883eefe5d3552 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:12:50.563993 25022 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d6ea032973024e708c5883eefe5d3552 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:12:50.566569 25022 raft_consensus.cc:504] T 00000000000000000000000000000000 P d6ea032973024e708c5883eefe5d3552 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6ea032973024e708c5883eefe5d3552" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 45139 } }
I20260807 18:12:50.567018 25022 leader_election.cc:302] T 00000000000000000000000000000000 P d6ea032973024e708c5883eefe5d3552 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d6ea032973024e708c5883eefe5d3552; no voters:
I20260807 18:12:50.567554 25022 leader_election.cc:288] T 00000000000000000000000000000000 P d6ea032973024e708c5883eefe5d3552 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:12:50.567718 25031 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d6ea032973024e708c5883eefe5d3552 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:12:50.568437 25031 raft_consensus.cc:686] T 00000000000000000000000000000000 P d6ea032973024e708c5883eefe5d3552 [term 1 LEADER]: Becoming Leader. State: Replica: d6ea032973024e708c5883eefe5d3552, State: Running, Role: LEADER
I20260807 18:12:50.568985 25031 consensus_queue.cc:237] T 00000000000000000000000000000000 P d6ea032973024e708c5883eefe5d3552 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6ea032973024e708c5883eefe5d3552" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 45139 } }
I20260807 18:12:50.569761 25022 sys_catalog.cc:564] T 00000000000000000000000000000000 P d6ea032973024e708c5883eefe5d3552 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:12:50.571976 25033 sys_catalog.cc:455] T 00000000000000000000000000000000 P d6ea032973024e708c5883eefe5d3552 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d6ea032973024e708c5883eefe5d3552" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6ea032973024e708c5883eefe5d3552" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 45139 } } }
I20260807 18:12:50.572436 25033 sys_catalog.cc:458] T 00000000000000000000000000000000 P d6ea032973024e708c5883eefe5d3552 [sys.catalog]: This master's current role is: LEADER
I20260807 18:12:50.572927 25037 sys_catalog.cc:455] T 00000000000000000000000000000000 P d6ea032973024e708c5883eefe5d3552 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d6ea032973024e708c5883eefe5d3552. Latest consensus state: current_term: 1 leader_uuid: "d6ea032973024e708c5883eefe5d3552" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6ea032973024e708c5883eefe5d3552" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 45139 } } }
I20260807 18:12:50.573283 25037 sys_catalog.cc:458] T 00000000000000000000000000000000 P d6ea032973024e708c5883eefe5d3552 [sys.catalog]: This master's current role is: LEADER
I20260807 18:12:50.573333 25045 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:12:50.579396 25045 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:12:50.588527 25045 catalog_manager.cc:1347] Generated new cluster ID: 0c32bb9b320f44f5a496b0331cde7f92
I20260807 18:12:50.588653 25045 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:12:50.615065 25045 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:12:50.616680 25045 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:12:50.628787 25045 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d6ea032973024e708c5883eefe5d3552: Generated new TSK 0
I20260807 18:12:50.629752 25045 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.154.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.154.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.154.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.1.keytab.
Entry for principal kudu/127.18.154.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.1.keytab.
I20260807 18:12:50.701709 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126328808105-19048-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126328808105-19048-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126328808105-19048-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126328808105-19048-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.154.1:0
--local_ip_for_outbound_sockets=127.18.154.1
--webserver_interface=127.18.154.1
--webserver_port=0
--tserver_master_addrs=127.18.154.62:45139
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.1.keytab
--principal=kudu/127.18.154.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.154.20:38131
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126328808105-19048-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126328808105-19048-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126328808105-19048-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:12:51.048852 25059 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:12:51.049228 25059 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:12:51.049445 25059 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:12:51.057394 25059 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:12:51.057507 25059 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:12:51.057713 25059 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.154.1
I20260807 18:12:51.067920 25059 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.154.20:38131
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126328808105-19048-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126328808105-19048-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.1.keytab
--principal=kudu/127.18.154.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.154.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126328808105-19048-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.154.1
--webserver_port=0
--tserver_master_addrs=127.18.154.62:45139
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.154.1
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126328808105-19048-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:12:51.069289 25059 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:12:51.070827 25059 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:12:51.084583 25065 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:12:51.084681 25071 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:12:51.084542 25068 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:12:51.086030 25059 server_base.cc:1034] running on GCE node
I20260807 18:12:51.086791 25059 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:12:51.087998 25059 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:12:51.089287 25059 hybrid_clock.cc:648] HybridClock initialized: now 1786126371089198 us; error 66 us; skew 500 ppm
I20260807 18:12:51.089722 25059 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:12:51 dist-test-slave-gkw7 krb5kdc[24916](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126371, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:12:51.093225 25059 init.cc:377] Logged in from keytab as kudu/127.18.154.1@KRBTEST.COM (short username kudu)
I20260807 18:12:51.094532 25059 fs_manager.cc:359] Metadata directory not provided
I20260807 18:12:51.094625 25059 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:12:51.094907 25059 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:12:51.097972 25059 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126328808105-19048-0/minicluster-data/ts-0/data/instance:
uuid: "d9b56136373a466ba7401b990a52b0f3"
format_stamp: "Formatted at 2026-08-07 18:12:51 on dist-test-slave-gkw7"
server_key: "39c33dd6f9998f7c141b0da56cb5521c"
server_key_iv: "e014c0bc970b1ec50f050f6b559f838e"
server_key_version: "encryptionkey@0"
I20260807 18:12:51.098773 25059 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126328808105-19048-0/minicluster-data/ts-0/wal/instance:
uuid: "d9b56136373a466ba7401b990a52b0f3"
format_stamp: "Formatted at 2026-08-07 18:12:51 on dist-test-slave-gkw7"
server_key: "39c33dd6f9998f7c141b0da56cb5521c"
server_key_iv: "e014c0bc970b1ec50f050f6b559f838e"
server_key_version: "encryptionkey@0"
I20260807 18:12:51.104276 25059 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260807 18:12:51.108084 25081 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:12:51.109622 25059 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260807 18:12:51.109807 25059 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126328808105-19048-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126328808105-19048-0/minicluster-data/ts-0/wal
uuid: "d9b56136373a466ba7401b990a52b0f3"
format_stamp: "Formatted at 2026-08-07 18:12:51 on dist-test-slave-gkw7"
server_key: "39c33dd6f9998f7c141b0da56cb5521c"
server_key_iv: "e014c0bc970b1ec50f050f6b559f838e"
server_key_version: "encryptionkey@0"
I20260807 18:12:51.110061 25059 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126328808105-19048-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126328808105-19048-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126328808105-19048-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:12:51.128152 25059 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:12:51.133003 25059 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:12:51.133318 25059 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:12:51.134389 25059 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:12:51.135823 25059 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:12:51.135954 25059 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:12:51.136101 25059 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:12:51.136200 25059 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:12:51.168848 25059 rpc_server.cc:307] RPC server started. Bound to: 127.18.154.1:36625
I20260807 18:12:51.168895 25241 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.154.1:36625 every 8 connection(s)
I20260807 18:12:51.169957 25059 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126328808105-19048-0/minicluster-data/ts-0/data/info.pb
I20260807 18:12:51.176332 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 25059
I20260807 18:12:51.176558 19048 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126328808105-19048-0/minicluster-data/ts-0/wal/instance
I20260807 18:12:51.177399 19048 external_mini_cluster.cc:1428] Setting key 13e917fcd3b3a5563e31278f469f7836
Aug 07 18:12:51 dist-test-slave-gkw7 krb5kdc[24916](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126371, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.1@KRBTEST.COM for kudu/127.18.154.62@KRBTEST.COM
I20260807 18:12:51.194682 25244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:51.172591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.1:50729 (local address 127.18.154.62:45139)
0807 18:12:51.173088 (+ 497us) server_negotiation.cc:206] Beginning negotiation
0807 18:12:51.173094 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:12:51.174181 (+ 1087us) server_negotiation.cc:407] Connection header received
0807 18:12:51.176137 (+ 1956us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:12:51.176142 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:12:51.176233 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:12:51.176335 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:12:51.178412 (+ 2077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:51.179300 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:51.180910 (+ 1610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:51.181324 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:51.181464 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:51.188675 (+ 7211us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:12:51.188680 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:12:51.188684 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:12:51.188726 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:12:51.191400 (+ 2674us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:12:51.192206 (+ 806us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:12:51.192212 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:12:51.192215 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:12:51.192339 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:12:51.192794 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:12:51.192798 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:12:51.192800 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:12:51.193074 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:12:51.193240 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0807 18:12:51.194060 (+ 820us) server_negotiation.cc:299] Negotiation successful
0807 18:12:51.194295 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":369,"thread_start_us":249,"threads_started":1}
I20260807 18:12:51.195719 25245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:51.173232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:45139 (local address 127.18.154.1:50729)
0807 18:12:51.174013 (+ 781us) negotiation.cc:107] Waiting for socket to connect
0807 18:12:51.174059 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 18:12:51.175808 (+ 1749us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:12:51.176633 (+ 825us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:12:51.176648 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:12:51.177446 (+ 798us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:12:51.178250 (+ 804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:12:51.178270 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:12:51.179471 (+ 1201us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:12:51.179476 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:12:51.180748 (+ 1272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:12:51.180760 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:12:51.181595 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:12:51.181603 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:12:51.181846 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:51.185929 (+ 4083us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:12:51.185954 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:12:51.188413 (+ 2459us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:12:51.191554 (+ 3141us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:12:51.191563 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:12:51.191582 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:12:51.192060 (+ 478us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:12:51.192455 (+ 395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:12:51.192461 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:12:51.192464 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:12:51.192653 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:12:51.193227 (+ 574us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:12:51.193235 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:12:51.193654 (+ 419us) client_negotiation.cc:769] Sending connection context
0807 18:12:51.193938 (+ 284us) client_negotiation.cc:240] Negotiation successful
0807 18:12:51.194529 (+ 591us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":548,"thread_start_us":367,"threads_started":1}
I20260807 18:12:51.197495 25242 heartbeater.cc:347] Connected to a master server at 127.18.154.62:45139
I20260807 18:12:51.197875 25242 heartbeater.cc:464] Registering TS with master...
I20260807 18:12:51.198585 25242 heartbeater.cc:511] Master 127.18.154.62:45139 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.18.154.2@KRBTEST.COM; defaulting to no policy
I20260807 18:12:51.200840 24978 ts_manager.cc:194] Registered new tserver with Master: d9b56136373a466ba7401b990a52b0f3 (127.18.154.1:36625)
I20260807 18:12:51.202418 24978 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.154.1@KRBTEST.COM'} at 127.18.154.1:50729
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.154.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.154.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.2.keytab.
Entry for principal kudu/127.18.154.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.2.keytab.
I20260807 18:12:51.232399 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126328808105-19048-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126328808105-19048-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126328808105-19048-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126328808105-19048-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.154.2:0
--local_ip_for_outbound_sockets=127.18.154.2
--webserver_interface=127.18.154.2
--webserver_port=0
--tserver_master_addrs=127.18.154.62:45139
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.2.keytab
--principal=kudu/127.18.154.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.154.20:38131
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126328808105-19048-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126328808105-19048-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126328808105-19048-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:12:51.587049 25249 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:12:51.587345 25249 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:12:51.587504 25249 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:12:51.594937 25249 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:12:51.595067 25249 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:12:51.595188 25249 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.154.2
I20260807 18:12:51.605309 25249 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.154.20:38131
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126328808105-19048-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126328808105-19048-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.2.keytab
--principal=kudu/127.18.154.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.154.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126328808105-19048-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.154.2
--webserver_port=0
--tserver_master_addrs=127.18.154.62:45139
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.154.2
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126328808105-19048-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:12:51.606931 25249 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:12:51.608426 25249 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:12:51.622759 25256 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:12:51.622997 25263 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:12:51.622824 25257 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:12:51.623663 25249 server_base.cc:1034] running on GCE node
I20260807 18:12:51.624382 25249 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:12:51.625490 25249 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:12:51.626775 25249 hybrid_clock.cc:648] HybridClock initialized: now 1786126371626723 us; error 35 us; skew 500 ppm
I20260807 18:12:51.627194 25249 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:12:51 dist-test-slave-gkw7 krb5kdc[24916](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126371, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:12:51.631069 25249 init.cc:377] Logged in from keytab as kudu/127.18.154.2@KRBTEST.COM (short username kudu)
I20260807 18:12:51.632457 25249 fs_manager.cc:359] Metadata directory not provided
I20260807 18:12:51.632560 25249 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:12:51.632917 25249 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:12:51.636600 25249 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126328808105-19048-0/minicluster-data/ts-1/data/instance:
uuid: "cad5fd4386b543daa117de0290ce1772"
format_stamp: "Formatted at 2026-08-07 18:12:51 on dist-test-slave-gkw7"
server_key: "211e03dd890d6b9b1b12de12dfaba681"
server_key_iv: "31770f6c7d7d0e9faee05e1f9dcb1382"
server_key_version: "encryptionkey@0"
I20260807 18:12:51.637642 25249 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126328808105-19048-0/minicluster-data/ts-1/wal/instance:
uuid: "cad5fd4386b543daa117de0290ce1772"
format_stamp: "Formatted at 2026-08-07 18:12:51 on dist-test-slave-gkw7"
server_key: "211e03dd890d6b9b1b12de12dfaba681"
server_key_iv: "31770f6c7d7d0e9faee05e1f9dcb1382"
server_key_version: "encryptionkey@0"
I20260807 18:12:51.643551 25249 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260807 18:12:51.647609 25269 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:12:51.649151 25249 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 18:12:51.649364 25249 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126328808105-19048-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126328808105-19048-0/minicluster-data/ts-1/wal
uuid: "cad5fd4386b543daa117de0290ce1772"
format_stamp: "Formatted at 2026-08-07 18:12:51 on dist-test-slave-gkw7"
server_key: "211e03dd890d6b9b1b12de12dfaba681"
server_key_iv: "31770f6c7d7d0e9faee05e1f9dcb1382"
server_key_version: "encryptionkey@0"
I20260807 18:12:51.649682 25249 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126328808105-19048-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126328808105-19048-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126328808105-19048-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:12:51.672175 25249 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:12:51.677037 25249 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:12:51.677373 25249 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:12:51.678529 25249 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:12:51.680059 25249 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:12:51.680181 25249 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:12:51.680327 25249 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:12:51.680413 25249 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:12:51.716279 25249 rpc_server.cc:307] RPC server started. Bound to: 127.18.154.2:46303
I20260807 18:12:51.716346 25433 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.154.2:46303 every 8 connection(s)
I20260807 18:12:51.717226 25249 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126328808105-19048-0/minicluster-data/ts-1/data/info.pb
I20260807 18:12:51.718382 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 25249
I20260807 18:12:51.718564 19048 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126328808105-19048-0/minicluster-data/ts-1/wal/instance
I20260807 18:12:51.719183 19048 external_mini_cluster.cc:1428] Setting key 0b3429f7a32741b13138f438f5818cab
Aug 07 18:12:51 dist-test-slave-gkw7 krb5kdc[24916](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126371, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.2@KRBTEST.COM for kudu/127.18.154.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.18.154.3@KRBTEST.COM; defaulting to no policy
I20260807 18:12:51.747315 25436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:51.720274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.2:32935 (local address 127.18.154.62:45139)
0807 18:12:51.721688 (+ 1414us) server_negotiation.cc:206] Beginning negotiation
0807 18:12:51.721697 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:12:51.726464 (+ 4767us) server_negotiation.cc:407] Connection header received
0807 18:12:51.729655 (+ 3191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:12:51.729662 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:12:51.729783 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:12:51.729917 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:12:51.731693 (+ 1776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:51.732953 (+ 1260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:51.734708 (+ 1755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:51.735554 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:51.735710 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:51.739789 (+ 4079us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:12:51.739800 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:12:51.739807 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:12:51.739859 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:12:51.743695 (+ 3836us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:12:51.744596 (+ 901us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:12:51.744602 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:12:51.744606 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:12:51.744778 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:12:51.745349 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:12:51.745353 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:12:51.745356 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:12:51.745761 (+ 405us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:12:51.745939 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0807 18:12:51.746751 (+ 812us) server_negotiation.cc:299] Negotiation successful
0807 18:12:51.746994 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1232,"thread_start_us":1101,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.154.3@KRBTEST.COM" created.
I20260807 18:12:51.748601 25437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:51.720948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:45139 (local address 127.18.154.2:32935)
0807 18:12:51.726223 (+ 5275us) negotiation.cc:107] Waiting for socket to connect
0807 18:12:51.726304 (+ 81us) client_negotiation.cc:174] Beginning negotiation
0807 18:12:51.729099 (+ 2795us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:12:51.730266 (+ 1167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:12:51.730276 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:12:51.730856 (+ 580us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:12:51.731491 (+ 635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:12:51.731510 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:12:51.733126 (+ 1616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:12:51.733130 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:12:51.734535 (+ 1405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:12:51.734548 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:12:51.735789 (+ 1241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:12:51.735795 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:12:51.736016 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:51.736816 (+ 800us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:12:51.736842 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:12:51.739505 (+ 2663us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:12:51.743883 (+ 4378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:12:51.743892 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:12:51.743909 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:12:51.744433 (+ 524us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:12:51.744955 (+ 522us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:12:51.744961 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:12:51.744964 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:12:51.745175 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:12:51.745940 (+ 765us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:12:51.745952 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:12:51.746344 (+ 392us) client_negotiation.cc:769] Sending connection context
0807 18:12:51.746592 (+ 248us) client_negotiation.cc:240] Negotiation successful
0807 18:12:51.747147 (+ 555us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4913,"thread_start_us":4675,"threads_started":1}
I20260807 18:12:51.750464 25434 heartbeater.cc:347] Connected to a master server at 127.18.154.62:45139
I20260807 18:12:51.750833 25434 heartbeater.cc:464] Registering TS with master...
I20260807 18:12:51.751708 25434 heartbeater.cc:511] Master 127.18.154.62:45139 requested a full tablet report, sending...
I20260807 18:12:51.753635 24978 ts_manager.cc:194] Registered new tserver with Master: cad5fd4386b543daa117de0290ce1772 (127.18.154.2:46303)
I20260807 18:12:51.754918 24978 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.154.2@KRBTEST.COM'} at 127.18.154.2:32935
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.154.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.3.keytab.
Entry for principal kudu/127.18.154.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.3.keytab.
I20260807 18:12:51.777041 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126328808105-19048-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126328808105-19048-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126328808105-19048-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126328808105-19048-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.154.3:0
--local_ip_for_outbound_sockets=127.18.154.3
--webserver_interface=127.18.154.3
--webserver_port=0
--tserver_master_addrs=127.18.154.62:45139
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.3.keytab
--principal=kudu/127.18.154.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.154.20:38131
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126328808105-19048-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126328808105-19048-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126328808105-19048-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:12:52.126756 25441 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:12:52.127051 25441 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:12:52.127214 25441 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:12:52.134589 25441 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:12:52.134712 25441 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:12:52.134899 25441 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.154.3
I20260807 18:12:52.144901 25441 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.154.20:38131
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126328808105-19048-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126328808105-19048-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.3.keytab
--principal=kudu/127.18.154.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.154.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126328808105-19048-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.154.3
--webserver_port=0
--tserver_master_addrs=127.18.154.62:45139
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.154.3
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126328808105-19048-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:12:52.146349 25441 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:12:52.147719 25441 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:12:52.161259 25449 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:12:52.161235 25451 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:12:52.161193 25453 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:12:52.162994 25441 server_base.cc:1034] running on GCE node
I20260807 18:12:52.163647 25441 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:12:52.164696 25441 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:12:52.165923 25441 hybrid_clock.cc:648] HybridClock initialized: now 1786126372165860 us; error 62 us; skew 500 ppm
I20260807 18:12:52.166266 25441 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:12:52 dist-test-slave-gkw7 krb5kdc[24916](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126372, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:12:52.169448 25441 init.cc:377] Logged in from keytab as kudu/127.18.154.3@KRBTEST.COM (short username kudu)
I20260807 18:12:52.170868 25441 fs_manager.cc:359] Metadata directory not provided
I20260807 18:12:52.171001 25441 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:12:52.171336 25441 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:12:52.174319 25441 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126328808105-19048-0/minicluster-data/ts-2/data/instance:
uuid: "61ee3fdaafec45f3bb551fc29f532b86"
format_stamp: "Formatted at 2026-08-07 18:12:52 on dist-test-slave-gkw7"
server_key: "76fd2ab31cfcf357962c0ffa60c6f1c0"
server_key_iv: "be344bbd1c3cbd2be42271d012dd2674"
server_key_version: "encryptionkey@0"
I20260807 18:12:52.175215 25441 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126328808105-19048-0/minicluster-data/ts-2/wal/instance:
uuid: "61ee3fdaafec45f3bb551fc29f532b86"
format_stamp: "Formatted at 2026-08-07 18:12:52 on dist-test-slave-gkw7"
server_key: "76fd2ab31cfcf357962c0ffa60c6f1c0"
server_key_iv: "be344bbd1c3cbd2be42271d012dd2674"
server_key_version: "encryptionkey@0"
I20260807 18:12:52.180693 25441 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260807 18:12:52.184319 25459 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:12:52.185834 25441 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:12:52.186007 25441 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126328808105-19048-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126328808105-19048-0/minicluster-data/ts-2/wal
uuid: "61ee3fdaafec45f3bb551fc29f532b86"
format_stamp: "Formatted at 2026-08-07 18:12:52 on dist-test-slave-gkw7"
server_key: "76fd2ab31cfcf357962c0ffa60c6f1c0"
server_key_iv: "be344bbd1c3cbd2be42271d012dd2674"
server_key_version: "encryptionkey@0"
I20260807 18:12:52.186252 25441 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126328808105-19048-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126328808105-19048-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126328808105-19048-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:12:52.204609 25441 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:12:52.205410 25242 heartbeater.cc:503] Master 127.18.154.62:45139 was elected leader, sending a full tablet report...
I20260807 18:12:52.209968 25441 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:12:52.210314 25441 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:12:52.211500 25441 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:12:52.213008 25441 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:12:52.213152 25441 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:12:52.213294 25441 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:12:52.213418 25441 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:12:52.250223 25441 rpc_server.cc:307] RPC server started. Bound to: 127.18.154.3:44895
I20260807 18:12:52.250268 25616 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.154.3:44895 every 8 connection(s)
I20260807 18:12:52.251200 25441 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126328808105-19048-0/minicluster-data/ts-2/data/info.pb
I20260807 18:12:52.252198 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 25441
I20260807 18:12:52.252400 19048 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126328808105-19048-0/minicluster-data/ts-2/wal/instance
I20260807 18:12:52.252979 19048 external_mini_cluster.cc:1428] Setting key 5cd7009936d6d97dbc0625d04aecdbea
Aug 07 18:12:52 dist-test-slave-gkw7 krb5kdc[24916](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126372, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.3@KRBTEST.COM for kudu/127.18.154.62@KRBTEST.COM
I20260807 18:12:52.273640 25619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:52.254438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.3:52309 (local address 127.18.154.62:45139)
0807 18:12:52.255168 (+ 730us) server_negotiation.cc:206] Beginning negotiation
0807 18:12:52.255173 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:12:52.256328 (+ 1155us) server_negotiation.cc:407] Connection header received
0807 18:12:52.258308 (+ 1980us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:12:52.258314 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:12:52.258407 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:12:52.258535 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:12:52.260093 (+ 1558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:52.260877 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:52.262603 (+ 1726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:52.263054 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:52.263199 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:52.266880 (+ 3681us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:12:52.266885 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:12:52.266888 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:12:52.266923 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:12:52.270258 (+ 3335us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:12:52.271033 (+ 775us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:12:52.271038 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:12:52.271042 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:12:52.271223 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:12:52.271676 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:12:52.271681 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:12:52.271684 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:12:52.271979 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:12:52.272159 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0807 18:12:52.272993 (+ 834us) server_negotiation.cc:299] Negotiation successful
0807 18:12:52.273224 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":622,"thread_start_us":526,"threads_started":1}
I20260807 18:12:52.274638 25621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:52.255274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:45139 (local address 127.18.154.3:52309)
0807 18:12:52.256175 (+ 901us) negotiation.cc:107] Waiting for socket to connect
0807 18:12:52.256223 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 18:12:52.258037 (+ 1814us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:12:52.258779 (+ 742us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:12:52.258787 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:12:52.259327 (+ 540us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:12:52.259910 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:12:52.259927 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:12:52.261041 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:12:52.261046 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:12:52.262443 (+ 1397us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:12:52.262456 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:12:52.263312 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:12:52.263320 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:12:52.263563 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:52.264273 (+ 710us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:12:52.264297 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:12:52.266647 (+ 2350us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:12:52.270406 (+ 3759us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:12:52.270417 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:12:52.270435 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:12:52.270901 (+ 466us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:12:52.271336 (+ 435us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:12:52.271341 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:12:52.271345 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:12:52.271550 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:12:52.272130 (+ 580us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:12:52.272139 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:12:52.272572 (+ 433us) client_negotiation.cc:769] Sending connection context
0807 18:12:52.272858 (+ 286us) client_negotiation.cc:240] Negotiation successful
0807 18:12:52.273460 (+ 602us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":587,"thread_start_us":341,"threads_started":1}
I20260807 18:12:52.276171 25617 heartbeater.cc:347] Connected to a master server at 127.18.154.62:45139
I20260807 18:12:52.276512 25617 heartbeater.cc:464] Registering TS with master...
I20260807 18:12:52.277204 25617 heartbeater.cc:511] Master 127.18.154.62:45139 requested a full tablet report, sending...
I20260807 18:12:52.278739 24978 ts_manager.cc:194] Registered new tserver with Master: 61ee3fdaafec45f3bb551fc29f532b86 (127.18.154.3:44895)
I20260807 18:12:52.279533 24978 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.154.3@KRBTEST.COM'} at 127.18.154.3:52309
I20260807 18:12:52.282179 19048 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 18:12:52.299892 25619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:52.287042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33818 (local address 127.18.154.62:45139)
0807 18:12:52.287264 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0807 18:12:52.287270 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:12:52.287820 (+ 550us) server_negotiation.cc:407] Connection header received
0807 18:12:52.287947 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:12:52.287951 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:12:52.288018 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:12:52.288102 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:12:52.289817 (+ 1715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:52.290666 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:52.291995 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:52.292427 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:52.292567 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:52.294965 (+ 2398us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:12:52.294971 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:12:52.294974 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:12:52.295011 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:12:52.297587 (+ 2576us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:12:52.298218 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:12:52.298222 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:12:52.298225 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:12:52.298356 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:12:52.298676 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:12:52.298680 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:12:52.298682 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:12:52.299004 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:12:52.299208 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0807 18:12:52.299483 (+ 275us) server_negotiation.cc:299] Negotiation successful
0807 18:12:52.299664 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260807 18:12:52.304373 24978 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33818:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 18:12:52.311326 24978 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:12:52.335376 25642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:52.325740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49128 (local address 127.18.154.1:36625)
0807 18:12:52.326435 (+ 695us) server_negotiation.cc:206] Beginning negotiation
0807 18:12:52.326443 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:12:52.326475 (+ 32us) server_negotiation.cc:407] Connection header received
0807 18:12:52.326694 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:12:52.326702 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:12:52.326976 (+ 274us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:12:52.327219 (+ 243us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:12:52.328225 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:52.330234 (+ 2009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:52.333084 (+ 2850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:52.334116 (+ 1032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:52.334279 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:52.334532 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0807 18:12:52.334856 (+ 324us) server_negotiation.cc:299] Negotiation successful
0807 18:12:52.335026 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":517,"thread_start_us":343,"threads_started":1}
I20260807 18:12:52.335487 25640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:52.325455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.1:36625 (local address 127.0.0.1:49128)
0807 18:12:52.326270 (+ 815us) negotiation.cc:107] Waiting for socket to connect
0807 18:12:52.326315 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 18:12:52.326476 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:12:52.327561 (+ 1085us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:12:52.327570 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:12:52.327604 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:12:52.327975 (+ 371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:12:52.327991 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:12:52.330394 (+ 2403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:12:52.330402 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:12:52.332856 (+ 2454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:12:52.332870 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:12:52.334372 (+ 1502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:12:52.334378 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:12:52.334584 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:52.334635 (+ 51us) client_negotiation.cc:769] Sending connection context
0807 18:12:52.334793 (+ 158us) client_negotiation.cc:240] Negotiation successful
0807 18:12:52.335019 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":486,"mutex_wait_us":1,"thread_start_us":322,"threads_started":1}
I20260807 18:12:52.338636 25647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:52.328629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.3:44895 (local address 127.0.0.1:40758)
0807 18:12:52.329281 (+ 652us) negotiation.cc:107] Waiting for socket to connect
0807 18:12:52.329304 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 18:12:52.329548 (+ 244us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:12:52.330367 (+ 819us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:12:52.330370 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:12:52.330386 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:12:52.331002 (+ 616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:12:52.331014 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:12:52.333019 (+ 2005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:12:52.333029 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:12:52.335763 (+ 2734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:12:52.335781 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:12:52.337717 (+ 1936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:12:52.337724 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:12:52.337911 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:52.337955 (+ 44us) client_negotiation.cc:769] Sending connection context
0807 18:12:52.338058 (+ 103us) client_negotiation.cc:240] Negotiation successful
0807 18:12:52.338197 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":489,"mutex_wait_us":99,"thread_start_us":310,"threads_started":1}
I20260807 18:12:52.338701 25645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:52.327892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40758 (local address 127.18.154.3:44895)
0807 18:12:52.328704 (+ 812us) server_negotiation.cc:206] Beginning negotiation
0807 18:12:52.328714 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:12:52.329421 (+ 707us) server_negotiation.cc:407] Connection header received
0807 18:12:52.329754 (+ 333us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:12:52.329765 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:12:52.329957 (+ 192us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:12:52.330093 (+ 136us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:12:52.331254 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:52.332877 (+ 1623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:52.335964 (+ 3087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:52.337458 (+ 1494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:52.337615 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:52.337871 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0807 18:12:52.338132 (+ 261us) server_negotiation.cc:299] Negotiation successful
0807 18:12:52.338319 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":577,"thread_start_us":379,"threads_started":1}
I20260807 18:12:52.339298 25643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:52.326916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.2:46303 (local address 127.0.0.1:37646)
0807 18:12:52.327536 (+ 620us) negotiation.cc:107] Waiting for socket to connect
0807 18:12:52.327560 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 18:12:52.327712 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:12:52.330350 (+ 2638us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:12:52.330354 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:12:52.330379 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:12:52.331008 (+ 629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:12:52.331016 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:12:52.332904 (+ 1888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:12:52.332910 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:12:52.336119 (+ 3209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:12:52.336141 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:12:52.338424 (+ 2283us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:12:52.338434 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:12:52.338639 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:52.338673 (+ 34us) client_negotiation.cc:769] Sending connection context
0807 18:12:52.338749 (+ 76us) client_negotiation.cc:240] Negotiation successful
0807 18:12:52.338906 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":462,"mutex_wait_us":223,"thread_start_us":317,"threads_started":1}
I20260807 18:12:52.339936 25644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:52.327240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37646 (local address 127.18.154.2:46303)
0807 18:12:52.329769 (+ 2529us) server_negotiation.cc:206] Beginning negotiation
0807 18:12:52.329777 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:12:52.329803 (+ 26us) server_negotiation.cc:407] Connection header received
0807 18:12:52.329891 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:12:52.329898 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:12:52.330109 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:12:52.330343 (+ 234us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:12:52.331247 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:52.332749 (+ 1502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:52.336676 (+ 3927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:52.338169 (+ 1493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:52.339197 (+ 1028us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:52.339393 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0807 18:12:52.339461 (+ 68us) server_negotiation.cc:299] Negotiation successful
0807 18:12:52.339618 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2350,"thread_start_us":2282,"threads_started":1}
I20260807 18:12:52.340039 25146 tablet_service.cc:1467] Processing CreateTablet for tablet e46f5c9f3ee74a0a86af5be1f3171328 (DEFAULT_TABLE table=test-table [id=7844ded5248c408d9aa2df705d948770]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:12:52.342515 25146 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e46f5c9f3ee74a0a86af5be1f3171328. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:12:52.342836 25530 tablet_service.cc:1467] Processing CreateTablet for tablet e46f5c9f3ee74a0a86af5be1f3171328 (DEFAULT_TABLE table=test-table [id=7844ded5248c408d9aa2df705d948770]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:12:52.343217 25343 tablet_service.cc:1467] Processing CreateTablet for tablet e46f5c9f3ee74a0a86af5be1f3171328 (DEFAULT_TABLE table=test-table [id=7844ded5248c408d9aa2df705d948770]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:12:52.344372 25530 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e46f5c9f3ee74a0a86af5be1f3171328. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:12:52.344390 25343 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e46f5c9f3ee74a0a86af5be1f3171328. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:12:52.356941 25648 tablet_bootstrap.cc:492] T e46f5c9f3ee74a0a86af5be1f3171328 P cad5fd4386b543daa117de0290ce1772: Bootstrap starting.
I20260807 18:12:52.359123 25649 tablet_bootstrap.cc:492] T e46f5c9f3ee74a0a86af5be1f3171328 P d9b56136373a466ba7401b990a52b0f3: Bootstrap starting.
I20260807 18:12:52.360595 25650 tablet_bootstrap.cc:492] T e46f5c9f3ee74a0a86af5be1f3171328 P 61ee3fdaafec45f3bb551fc29f532b86: Bootstrap starting.
I20260807 18:12:52.361733 25648 tablet_bootstrap.cc:654] T e46f5c9f3ee74a0a86af5be1f3171328 P cad5fd4386b543daa117de0290ce1772: Neither blocks nor log segments found. Creating new log.
I20260807 18:12:52.363168 25648 log.cc:824] T e46f5c9f3ee74a0a86af5be1f3171328 P cad5fd4386b543daa117de0290ce1772: Log is configured to *not* fsync() on all Append() calls
I20260807 18:12:52.363677 25650 tablet_bootstrap.cc:654] T e46f5c9f3ee74a0a86af5be1f3171328 P 61ee3fdaafec45f3bb551fc29f532b86: Neither blocks nor log segments found. Creating new log.
I20260807 18:12:52.364006 25649 tablet_bootstrap.cc:654] T e46f5c9f3ee74a0a86af5be1f3171328 P d9b56136373a466ba7401b990a52b0f3: Neither blocks nor log segments found. Creating new log.
I20260807 18:12:52.364975 25650 log.cc:824] T e46f5c9f3ee74a0a86af5be1f3171328 P 61ee3fdaafec45f3bb551fc29f532b86: Log is configured to *not* fsync() on all Append() calls
I20260807 18:12:52.365110 25649 log.cc:824] T e46f5c9f3ee74a0a86af5be1f3171328 P d9b56136373a466ba7401b990a52b0f3: Log is configured to *not* fsync() on all Append() calls
I20260807 18:12:52.366812 25648 tablet_bootstrap.cc:492] T e46f5c9f3ee74a0a86af5be1f3171328 P cad5fd4386b543daa117de0290ce1772: No bootstrap required, opened a new log
I20260807 18:12:52.367112 25648 ts_tablet_manager.cc:1397] T e46f5c9f3ee74a0a86af5be1f3171328 P cad5fd4386b543daa117de0290ce1772: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.003s
I20260807 18:12:52.368054 25649 tablet_bootstrap.cc:492] T e46f5c9f3ee74a0a86af5be1f3171328 P d9b56136373a466ba7401b990a52b0f3: No bootstrap required, opened a new log
I20260807 18:12:52.368342 25650 tablet_bootstrap.cc:492] T e46f5c9f3ee74a0a86af5be1f3171328 P 61ee3fdaafec45f3bb551fc29f532b86: No bootstrap required, opened a new log
I20260807 18:12:52.368351 25649 ts_tablet_manager.cc:1397] T e46f5c9f3ee74a0a86af5be1f3171328 P d9b56136373a466ba7401b990a52b0f3: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.006s
I20260807 18:12:52.368599 25650 ts_tablet_manager.cc:1397] T e46f5c9f3ee74a0a86af5be1f3171328 P 61ee3fdaafec45f3bb551fc29f532b86: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260807 18:12:52.375742 25649 raft_consensus.cc:348] T e46f5c9f3ee74a0a86af5be1f3171328 P d9b56136373a466ba7401b990a52b0f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9b56136373a466ba7401b990a52b0f3" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 36625 } } peers { permanent_uuid: "cad5fd4386b543daa117de0290ce1772" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 46303 } } peers { permanent_uuid: "61ee3fdaafec45f3bb551fc29f532b86" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 44895 } }
I20260807 18:12:52.376252 25649 raft_consensus.cc:374] T e46f5c9f3ee74a0a86af5be1f3171328 P d9b56136373a466ba7401b990a52b0f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:12:52.376365 25649 raft_consensus.cc:729] T e46f5c9f3ee74a0a86af5be1f3171328 P d9b56136373a466ba7401b990a52b0f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9b56136373a466ba7401b990a52b0f3, State: Initialized, Role: FOLLOWER
I20260807 18:12:52.377132 25649 consensus_queue.cc:260] T e46f5c9f3ee74a0a86af5be1f3171328 P d9b56136373a466ba7401b990a52b0f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9b56136373a466ba7401b990a52b0f3" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 36625 } } peers { permanent_uuid: "cad5fd4386b543daa117de0290ce1772" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 46303 } } peers { permanent_uuid: "61ee3fdaafec45f3bb551fc29f532b86" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 44895 } }
I20260807 18:12:52.378304 25648 raft_consensus.cc:348] T e46f5c9f3ee74a0a86af5be1f3171328 P cad5fd4386b543daa117de0290ce1772 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9b56136373a466ba7401b990a52b0f3" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 36625 } } peers { permanent_uuid: "cad5fd4386b543daa117de0290ce1772" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 46303 } } peers { permanent_uuid: "61ee3fdaafec45f3bb551fc29f532b86" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 44895 } }
I20260807 18:12:52.378762 25648 raft_consensus.cc:374] T e46f5c9f3ee74a0a86af5be1f3171328 P cad5fd4386b543daa117de0290ce1772 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:12:52.378851 25648 raft_consensus.cc:729] T e46f5c9f3ee74a0a86af5be1f3171328 P cad5fd4386b543daa117de0290ce1772 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cad5fd4386b543daa117de0290ce1772, State: Initialized, Role: FOLLOWER
I20260807 18:12:52.379245 25649 ts_tablet_manager.cc:1428] T e46f5c9f3ee74a0a86af5be1f3171328 P d9b56136373a466ba7401b990a52b0f3: Time spent starting tablet: real 0.011s user 0.008s sys 0.004s
I20260807 18:12:52.379437 25650 raft_consensus.cc:348] T e46f5c9f3ee74a0a86af5be1f3171328 P 61ee3fdaafec45f3bb551fc29f532b86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9b56136373a466ba7401b990a52b0f3" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 36625 } } peers { permanent_uuid: "cad5fd4386b543daa117de0290ce1772" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 46303 } } peers { permanent_uuid: "61ee3fdaafec45f3bb551fc29f532b86" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 44895 } }
I20260807 18:12:52.379619 25648 consensus_queue.cc:260] T e46f5c9f3ee74a0a86af5be1f3171328 P cad5fd4386b543daa117de0290ce1772 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9b56136373a466ba7401b990a52b0f3" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 36625 } } peers { permanent_uuid: "cad5fd4386b543daa117de0290ce1772" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 46303 } } peers { permanent_uuid: "61ee3fdaafec45f3bb551fc29f532b86" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 44895 } }
I20260807 18:12:52.379897 25650 raft_consensus.cc:374] T e46f5c9f3ee74a0a86af5be1f3171328 P 61ee3fdaafec45f3bb551fc29f532b86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:12:52.379994 25650 raft_consensus.cc:729] T e46f5c9f3ee74a0a86af5be1f3171328 P 61ee3fdaafec45f3bb551fc29f532b86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61ee3fdaafec45f3bb551fc29f532b86, State: Initialized, Role: FOLLOWER
I20260807 18:12:52.380744 25650 consensus_queue.cc:260] T e46f5c9f3ee74a0a86af5be1f3171328 P 61ee3fdaafec45f3bb551fc29f532b86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9b56136373a466ba7401b990a52b0f3" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 36625 } } peers { permanent_uuid: "cad5fd4386b543daa117de0290ce1772" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 46303 } } peers { permanent_uuid: "61ee3fdaafec45f3bb551fc29f532b86" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 44895 } }
I20260807 18:12:52.382157 25617 heartbeater.cc:503] Master 127.18.154.62:45139 was elected leader, sending a full tablet report...
I20260807 18:12:52.383111 25650 ts_tablet_manager.cc:1428] T e46f5c9f3ee74a0a86af5be1f3171328 P 61ee3fdaafec45f3bb551fc29f532b86: Time spent starting tablet: real 0.014s user 0.013s sys 0.000s
I20260807 18:12:52.385160 25434 heartbeater.cc:503] Master 127.18.154.62:45139 was elected leader, sending a full tablet report...
I20260807 18:12:52.385916 25648 ts_tablet_manager.cc:1428] T e46f5c9f3ee74a0a86af5be1f3171328 P cad5fd4386b543daa117de0290ce1772: Time spent starting tablet: real 0.019s user 0.011s sys 0.008s
W20260807 18:12:52.422437 25243 tablet.cc:2369] T e46f5c9f3ee74a0a86af5be1f3171328 P d9b56136373a466ba7401b990a52b0f3: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 18:12:52.469348 25435 tablet.cc:2369] T e46f5c9f3ee74a0a86af5be1f3171328 P cad5fd4386b543daa117de0290ce1772: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 18:12:52.502887 25618 tablet.cc:2369] T e46f5c9f3ee74a0a86af5be1f3171328 P 61ee3fdaafec45f3bb551fc29f532b86: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:12:52.562045 25654 raft_consensus.cc:482] T e46f5c9f3ee74a0a86af5be1f3171328 P d9b56136373a466ba7401b990a52b0f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:12:52.562412 25654 raft_consensus.cc:504] T e46f5c9f3ee74a0a86af5be1f3171328 P d9b56136373a466ba7401b990a52b0f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9b56136373a466ba7401b990a52b0f3" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 36625 } } peers { permanent_uuid: "cad5fd4386b543daa117de0290ce1772" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 46303 } } peers { permanent_uuid: "61ee3fdaafec45f3bb551fc29f532b86" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 44895 } }
I20260807 18:12:52.564200 25654 leader_election.cc:288] T e46f5c9f3ee74a0a86af5be1f3171328 P d9b56136373a466ba7401b990a52b0f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cad5fd4386b543daa117de0290ce1772 (127.18.154.2:46303), 61ee3fdaafec45f3bb551fc29f532b86 (127.18.154.3:44895)
I20260807 18:12:52.572162 25645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:52.565861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.1:52257 (local address 127.18.154.3:44895)
0807 18:12:52.566073 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0807 18:12:52.566082 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:12:52.566280 (+ 198us) server_negotiation.cc:407] Connection header received
0807 18:12:52.566399 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:12:52.566405 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:12:52.566493 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:12:52.566627 (+ 134us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:12:52.567201 (+ 574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:52.567938 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:52.569962 (+ 2024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:52.571093 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:52.571315 (+ 222us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:52.571395 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0807 18:12:52.571779 (+ 384us) server_negotiation.cc:299] Negotiation successful
0807 18:12:52.571933 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260807 18:12:52.572307 25660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:52.565487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.3:44895 (local address 127.18.154.1:52257)
0807 18:12:52.566166 (+ 679us) negotiation.cc:107] Waiting for socket to connect
0807 18:12:52.566180 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 18:12:52.566298 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:12:52.566631 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:12:52.566634 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:12:52.566652 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:12:52.567033 (+ 381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:12:52.567041 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:12:52.568092 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:12:52.568097 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:12:52.569792 (+ 1695us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:12:52.569805 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:12:52.571346 (+ 1541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:12:52.571352 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:12:52.571552 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:52.571587 (+ 35us) client_negotiation.cc:769] Sending connection context
0807 18:12:52.571739 (+ 152us) client_negotiation.cc:240] Negotiation successful
0807 18:12:52.571890 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":580,"mutex_wait_us":16,"thread_start_us":410,"threads_started":1}
I20260807 18:12:52.572340 25644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:52.564683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.1:48353 (local address 127.18.154.2:46303)
0807 18:12:52.564890 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0807 18:12:52.564896 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:12:52.565596 (+ 700us) server_negotiation.cc:407] Connection header received
0807 18:12:52.565911 (+ 315us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:12:52.565918 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:12:52.566009 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:12:52.566107 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:12:52.567048 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:52.567855 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:52.570062 (+ 2207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:52.571339 (+ 1277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:52.571498 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:52.571595 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0807 18:12:52.571943 (+ 348us) server_negotiation.cc:299] Negotiation successful
0807 18:12:52.572090 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260807 18:12:52.573086 25659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:52.564680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.2:46303 (local address 127.18.154.1:48353)
0807 18:12:52.565403 (+ 723us) negotiation.cc:107] Waiting for socket to connect
0807 18:12:52.565447 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 18:12:52.565681 (+ 234us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:12:52.566355 (+ 674us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:12:52.566362 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:12:52.566392 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:12:52.566882 (+ 490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:12:52.566893 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:12:52.567999 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:12:52.568008 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:12:52.569896 (+ 1888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:12:52.569909 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:12:52.571558 (+ 1649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:12:52.571564 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:12:52.571773 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:52.571801 (+ 28us) client_negotiation.cc:769] Sending connection context
0807 18:12:52.572471 (+ 670us) client_negotiation.cc:240] Negotiation successful
0807 18:12:52.572650 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":609,"mutex_wait_us":4,"thread_start_us":404,"threads_started":1}
I20260807 18:12:52.573489 25557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e46f5c9f3ee74a0a86af5be1f3171328" candidate_uuid: "d9b56136373a466ba7401b990a52b0f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61ee3fdaafec45f3bb551fc29f532b86" is_pre_election: true
I20260807 18:12:52.574267 25557 raft_consensus.cc:2454] T e46f5c9f3ee74a0a86af5be1f3171328 P 61ee3fdaafec45f3bb551fc29f532b86 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9b56136373a466ba7401b990a52b0f3 in term 0.
I20260807 18:12:52.574254 25374 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e46f5c9f3ee74a0a86af5be1f3171328" candidate_uuid: "d9b56136373a466ba7401b990a52b0f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cad5fd4386b543daa117de0290ce1772" is_pre_election: true
I20260807 18:12:52.574867 25374 raft_consensus.cc:2454] T e46f5c9f3ee74a0a86af5be1f3171328 P cad5fd4386b543daa117de0290ce1772 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9b56136373a466ba7401b990a52b0f3 in term 0.
I20260807 18:12:52.575083 25085 leader_election.cc:302] T e46f5c9f3ee74a0a86af5be1f3171328 P d9b56136373a466ba7401b990a52b0f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 61ee3fdaafec45f3bb551fc29f532b86, d9b56136373a466ba7401b990a52b0f3; no voters:
I20260807 18:12:52.575585 25654 raft_consensus.cc:2781] T e46f5c9f3ee74a0a86af5be1f3171328 P d9b56136373a466ba7401b990a52b0f3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:12:52.575762 25654 raft_consensus.cc:482] T e46f5c9f3ee74a0a86af5be1f3171328 P d9b56136373a466ba7401b990a52b0f3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:12:52.575899 25654 raft_consensus.cc:3037] T e46f5c9f3ee74a0a86af5be1f3171328 P d9b56136373a466ba7401b990a52b0f3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:12:52.578387 25654 raft_consensus.cc:504] T e46f5c9f3ee74a0a86af5be1f3171328 P d9b56136373a466ba7401b990a52b0f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9b56136373a466ba7401b990a52b0f3" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 36625 } } peers { permanent_uuid: "cad5fd4386b543daa117de0290ce1772" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 46303 } } peers { permanent_uuid: "61ee3fdaafec45f3bb551fc29f532b86" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 44895 } }
I20260807 18:12:52.579041 25654 leader_election.cc:288] T e46f5c9f3ee74a0a86af5be1f3171328 P d9b56136373a466ba7401b990a52b0f3 [CANDIDATE]: Term 1 election: Requested vote from peers cad5fd4386b543daa117de0290ce1772 (127.18.154.2:46303), 61ee3fdaafec45f3bb551fc29f532b86 (127.18.154.3:44895)
I20260807 18:12:52.579496 25374 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e46f5c9f3ee74a0a86af5be1f3171328" candidate_uuid: "d9b56136373a466ba7401b990a52b0f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cad5fd4386b543daa117de0290ce1772"
I20260807 18:12:52.579656 25557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e46f5c9f3ee74a0a86af5be1f3171328" candidate_uuid: "d9b56136373a466ba7401b990a52b0f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61ee3fdaafec45f3bb551fc29f532b86"
I20260807 18:12:52.579717 25374 raft_consensus.cc:3037] T e46f5c9f3ee74a0a86af5be1f3171328 P cad5fd4386b543daa117de0290ce1772 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:12:52.579890 25557 raft_consensus.cc:3037] T e46f5c9f3ee74a0a86af5be1f3171328 P 61ee3fdaafec45f3bb551fc29f532b86 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:12:52.582247 25557 raft_consensus.cc:2454] T e46f5c9f3ee74a0a86af5be1f3171328 P 61ee3fdaafec45f3bb551fc29f532b86 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9b56136373a466ba7401b990a52b0f3 in term 1.
I20260807 18:12:52.582290 25374 raft_consensus.cc:2454] T e46f5c9f3ee74a0a86af5be1f3171328 P cad5fd4386b543daa117de0290ce1772 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9b56136373a466ba7401b990a52b0f3 in term 1.
I20260807 18:12:52.582692 25085 leader_election.cc:302] T e46f5c9f3ee74a0a86af5be1f3171328 P d9b56136373a466ba7401b990a52b0f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 61ee3fdaafec45f3bb551fc29f532b86, d9b56136373a466ba7401b990a52b0f3; no voters:
I20260807 18:12:52.583037 25654 raft_consensus.cc:2781] T e46f5c9f3ee74a0a86af5be1f3171328 P d9b56136373a466ba7401b990a52b0f3 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:12:52.583567 25654 raft_consensus.cc:686] T e46f5c9f3ee74a0a86af5be1f3171328 P d9b56136373a466ba7401b990a52b0f3 [term 1 LEADER]: Becoming Leader. State: Replica: d9b56136373a466ba7401b990a52b0f3, State: Running, Role: LEADER
I20260807 18:12:52.584117 25654 consensus_queue.cc:237] T e46f5c9f3ee74a0a86af5be1f3171328 P d9b56136373a466ba7401b990a52b0f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9b56136373a466ba7401b990a52b0f3" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 36625 } } peers { permanent_uuid: "cad5fd4386b543daa117de0290ce1772" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 46303 } } peers { permanent_uuid: "61ee3fdaafec45f3bb551fc29f532b86" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 44895 } }
I20260807 18:12:52.589437 24977 catalog_manager.cc:5495] T e46f5c9f3ee74a0a86af5be1f3171328 P d9b56136373a466ba7401b990a52b0f3 reported cstate change: term changed from 0 to 1, leader changed from <none> to d9b56136373a466ba7401b990a52b0f3 (127.18.154.1). New cstate: current_term: 1 leader_uuid: "d9b56136373a466ba7401b990a52b0f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9b56136373a466ba7401b990a52b0f3" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 36625 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cad5fd4386b543daa117de0290ce1772" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 46303 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "61ee3fdaafec45f3bb551fc29f532b86" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 44895 } health_report { overall_health: UNKNOWN } } }
I20260807 18:12:52.642855 25642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:52.637059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49134 (local address 127.18.154.1:36625)
0807 18:12:52.637315 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0807 18:12:52.637320 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:12:52.637342 (+ 22us) server_negotiation.cc:407] Connection header received
0807 18:12:52.637614 (+ 272us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:12:52.637619 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:12:52.637711 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:12:52.637865 (+ 154us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:12:52.638271 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:52.639147 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:52.640502 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:52.640959 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:52.641074 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:52.641632 (+ 558us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:12:52.642259 (+ 627us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:12:52.642385 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0807 18:12:52.642555 (+ 170us) server_negotiation.cc:299] Negotiation successful
0807 18:12:52.642632 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105}
I20260807 18:12:52.651929 25374 raft_consensus.cc:1261] T e46f5c9f3ee74a0a86af5be1f3171328 P cad5fd4386b543daa117de0290ce1772 [term 1 FOLLOWER]: Refusing update from remote peer d9b56136373a466ba7401b990a52b0f3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:12:52.651917 25557 raft_consensus.cc:1261] T e46f5c9f3ee74a0a86af5be1f3171328 P 61ee3fdaafec45f3bb551fc29f532b86 [term 1 FOLLOWER]: Refusing update from remote peer d9b56136373a466ba7401b990a52b0f3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:12:52.653025 25661 consensus_queue.cc:1035] T e46f5c9f3ee74a0a86af5be1f3171328 P d9b56136373a466ba7401b990a52b0f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61ee3fdaafec45f3bb551fc29f532b86" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 44895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:12:52.653446 25654 consensus_queue.cc:1035] T e46f5c9f3ee74a0a86af5be1f3171328 P d9b56136373a466ba7401b990a52b0f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cad5fd4386b543daa117de0290ce1772" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 46303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:12:52.665427 25665 mvcc.cc:205] Tried to move back new op lower bound from 7315973622370701312 to 7315973622108954624. Current Snapshot: MvccSnapshot[applied={T|T < 7315973622370701312}]
I20260807 18:12:52.682813 24978 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33818:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:12:52.683128 24978 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33818:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:12:52.687402 24978 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d6ea032973024e708c5883eefe5d3552: Sending DeleteTablet for 3 replicas of tablet e46f5c9f3ee74a0a86af5be1f3171328
I20260807 18:12:52.688738 25146 tablet_service.cc:1514] Processing DeleteTablet for tablet e46f5c9f3ee74a0a86af5be1f3171328 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:12:52 UTC) from {username='kudu', principal='kudu/127.18.154.62@KRBTEST.COM'} at 127.0.0.1:49128
I20260807 18:12:52.688865 25343 tablet_service.cc:1514] Processing DeleteTablet for tablet e46f5c9f3ee74a0a86af5be1f3171328 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:12:52 UTC) from {username='kudu', principal='kudu/127.18.154.62@KRBTEST.COM'} at 127.0.0.1:37646
I20260807 18:12:52.689158 25530 tablet_service.cc:1514] Processing DeleteTablet for tablet e46f5c9f3ee74a0a86af5be1f3171328 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:12:52 UTC) from {username='kudu', principal='kudu/127.18.154.62@KRBTEST.COM'} at 127.0.0.1:40758
I20260807 18:12:52.691268 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 25059
I20260807 18:12:52.701936 25679 tablet_replica.cc:330] T e46f5c9f3ee74a0a86af5be1f3171328 P d9b56136373a466ba7401b990a52b0f3: stopping tablet replica
I20260807 18:12:52.703416 25679 raft_consensus.cc:2229] T e46f5c9f3ee74a0a86af5be1f3171328 P d9b56136373a466ba7401b990a52b0f3 [term 1 LEADER]: Raft consensus shutting down.
I20260807 18:12:52.704124 25678 tablet_replica.cc:330] T e46f5c9f3ee74a0a86af5be1f3171328 P cad5fd4386b543daa117de0290ce1772: stopping tablet replica
I20260807 18:12:52.704471 25679 raft_consensus.cc:2258] T e46f5c9f3ee74a0a86af5be1f3171328 P d9b56136373a466ba7401b990a52b0f3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:12:52.704589 25678 raft_consensus.cc:2229] T e46f5c9f3ee74a0a86af5be1f3171328 P cad5fd4386b543daa117de0290ce1772 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:12:52.705175 25678 raft_consensus.cc:2258] T e46f5c9f3ee74a0a86af5be1f3171328 P cad5fd4386b543daa117de0290ce1772 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:12:52.707540 25679 ts_tablet_manager.cc:1905] T e46f5c9f3ee74a0a86af5be1f3171328 P d9b56136373a466ba7401b990a52b0f3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:12:52.707532 25678 ts_tablet_manager.cc:1905] T e46f5c9f3ee74a0a86af5be1f3171328 P cad5fd4386b543daa117de0290ce1772: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:12:52.714998 25678 ts_tablet_manager.cc:1918] T e46f5c9f3ee74a0a86af5be1f3171328 P cad5fd4386b543daa117de0290ce1772: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:12:52.715210 25678 log.cc:1196] T e46f5c9f3ee74a0a86af5be1f3171328 P cad5fd4386b543daa117de0290ce1772: Deleting WAL directory at /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126328808105-19048-0/minicluster-data/ts-1/wal/wals/e46f5c9f3ee74a0a86af5be1f3171328
I20260807 18:12:52.715571 25679 ts_tablet_manager.cc:1918] T e46f5c9f3ee74a0a86af5be1f3171328 P d9b56136373a466ba7401b990a52b0f3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:12:52.715938 25679 log.cc:1196] T e46f5c9f3ee74a0a86af5be1f3171328 P d9b56136373a466ba7401b990a52b0f3: Deleting WAL directory at /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126328808105-19048-0/minicluster-data/ts-0/wal/wals/e46f5c9f3ee74a0a86af5be1f3171328
I20260807 18:12:52.716634 25679 ts_tablet_manager.cc:1939] T e46f5c9f3ee74a0a86af5be1f3171328 P d9b56136373a466ba7401b990a52b0f3: Deleting consensus metadata
I20260807 18:12:52.718719 24960 catalog_manager.cc:4841] TS d9b56136373a466ba7401b990a52b0f3 (127.18.154.1:36625): tablet e46f5c9f3ee74a0a86af5be1f3171328 (table test-table [id=7844ded5248c408d9aa2df705d948770]) successfully deleted
I20260807 18:12:52.720252 25680 tablet_replica.cc:330] T e46f5c9f3ee74a0a86af5be1f3171328 P 61ee3fdaafec45f3bb551fc29f532b86: stopping tablet replica
I20260807 18:12:52.720717 25680 raft_consensus.cc:2229] T e46f5c9f3ee74a0a86af5be1f3171328 P 61ee3fdaafec45f3bb551fc29f532b86 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:12:52.721263 25680 raft_consensus.cc:2258] T e46f5c9f3ee74a0a86af5be1f3171328 P 61ee3fdaafec45f3bb551fc29f532b86 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:12:52.722119 25678 ts_tablet_manager.cc:1939] T e46f5c9f3ee74a0a86af5be1f3171328 P cad5fd4386b543daa117de0290ce1772: Deleting consensus metadata
I20260807 18:12:52.724009 24960 catalog_manager.cc:4841] TS cad5fd4386b543daa117de0290ce1772 (127.18.154.2:46303): tablet e46f5c9f3ee74a0a86af5be1f3171328 (table test-table [id=7844ded5248c408d9aa2df705d948770]) successfully deleted
Aug 07 18:12:52 dist-test-slave-gkw7 krb5kdc[24916](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126369, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.154.1@KRBTEST.COM
I20260807 18:12:52.731789 25680 ts_tablet_manager.cc:1905] T e46f5c9f3ee74a0a86af5be1f3171328 P 61ee3fdaafec45f3bb551fc29f532b86: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:12:52.738998 25642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:52.692524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49136 (local address 127.18.154.1:36625)
0807 18:12:52.692705 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0807 18:12:52.692713 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:12:52.716265 (+ 23552us) server_negotiation.cc:407] Connection header received
0807 18:12:52.716815 (+ 550us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:12:52.716821 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:12:52.716926 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:12:52.717021 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:12:52.719824 (+ 2803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:52.721054 (+ 1230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:52.723548 (+ 2494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:52.724262 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:52.724367 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:52.729957 (+ 5590us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:12:52.729976 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:12:52.729997 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:12:52.730061 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:12:52.734367 (+ 4306us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:12:52.735385 (+ 1018us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:12:52.735395 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:12:52.735399 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:12:52.735581 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:12:52.736055 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:12:52.736061 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:12:52.736064 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:12:52.736596 (+ 532us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:12:52.736914 (+ 318us) server_negotiation.cc:1035] Waiting for connection context
0807 18:12:52.738249 (+ 1335us) server_negotiation.cc:299] Negotiation successful
0807 18:12:52.738520 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260807 18:12:52.739460 25680 ts_tablet_manager.cc:1918] T e46f5c9f3ee74a0a86af5be1f3171328 P 61ee3fdaafec45f3bb551fc29f532b86: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:12:52.739629 25680 log.cc:1196] T e46f5c9f3ee74a0a86af5be1f3171328 P 61ee3fdaafec45f3bb551fc29f532b86: Deleting WAL directory at /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786126328808105-19048-0/minicluster-data/ts-2/wal/wals/e46f5c9f3ee74a0a86af5be1f3171328
I20260807 18:12:52.740166 25237 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:12:52.742009 25680 ts_tablet_manager.cc:1939] T e46f5c9f3ee74a0a86af5be1f3171328 P 61ee3fdaafec45f3bb551fc29f532b86: Deleting consensus metadata
I20260807 18:12:52.744374 24960 catalog_manager.cc:4841] TS 61ee3fdaafec45f3bb551fc29f532b86 (127.18.154.3:44895): tablet e46f5c9f3ee74a0a86af5be1f3171328 (table test-table [id=7844ded5248c408d9aa2df705d948770]) successfully deleted
I20260807 18:12:52.843562 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 25059
I20260807 18:12:52.868969 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 25249
Aug 07 18:12:52 dist-test-slave-gkw7 krb5kdc[24916](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126369, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.154.2@KRBTEST.COM
I20260807 18:12:52.885761 25644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:52.869861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37650 (local address 127.18.154.2:46303)
0807 18:12:52.870054 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0807 18:12:52.870063 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:12:52.870208 (+ 145us) server_negotiation.cc:407] Connection header received
0807 18:12:52.870435 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:12:52.870441 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:12:52.870590 (+ 149us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:12:52.870738 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:12:52.872074 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:52.872860 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:52.874365 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:52.874805 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:52.874948 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:52.879350 (+ 4402us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:12:52.879366 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:12:52.879382 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:12:52.879420 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:12:52.882714 (+ 3294us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:12:52.883650 (+ 936us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:12:52.883659 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:12:52.883661 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:12:52.883796 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:12:52.884186 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:12:52.884190 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:12:52.884193 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:12:52.884622 (+ 429us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:12:52.884862 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0807 18:12:52.885236 (+ 374us) server_negotiation.cc:299] Negotiation successful
0807 18:12:52.885424 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260807 18:12:52.886698 25428 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:12:52.981956 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 25249
I20260807 18:12:53.007372 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 25441
Aug 07 18:12:53 dist-test-slave-gkw7 krb5kdc[24916](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126369, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.154.3@KRBTEST.COM
I20260807 18:12:53.023062 25645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:53.008203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40766 (local address 127.18.154.3:44895)
0807 18:12:53.008390 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0807 18:12:53.008395 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:12:53.008415 (+ 20us) server_negotiation.cc:407] Connection header received
0807 18:12:53.008559 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:12:53.008562 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:12:53.008630 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:12:53.008724 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:12:53.010144 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:53.010917 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:53.012218 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:53.012623 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:53.012727 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:53.017262 (+ 4535us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:12:53.017276 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:12:53.017294 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:12:53.017333 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:12:53.020308 (+ 2975us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:12:53.020944 (+ 636us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:12:53.020952 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:12:53.020955 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:12:53.021126 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:12:53.021647 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:12:53.021652 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:12:53.021654 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:12:53.022058 (+ 404us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:12:53.022302 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0807 18:12:53.022595 (+ 293us) server_negotiation.cc:299] Negotiation successful
0807 18:12:53.022776 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260807 18:12:53.023803 25610 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:12:53.116357 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 25441
I20260807 18:12:53.140225 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 24935
I20260807 18:12:53.141507 25016 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:12:53.251207 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 24935
2026-08-07T18:12:53Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (4693 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-07T18:12:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:12:53Z Disabled control of system clock
I20260807 18:12:53.338801 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126328808105-19048-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126328808105-19048-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126328808105-19048-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.154.62:34417
--webserver_interface=127.18.154.62
--webserver_port=0
--builtin_ntp_servers=127.18.154.20:40235
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.154.62:34417
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260807 18:12:53.682289 25692 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:12:53.682603 25692 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:12:53.682690 25692 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:12:53.690013 25692 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:12:53.690127 25692 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:12:53.690169 25692 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:12:53.690237 25692 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:12:53.690284 25692 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:12:53.699416 25692 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.154.20:40235
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126328808105-19048-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126328808105-19048-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.154.62:34417
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.154.62:34417
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.154.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126328808105-19048-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:12:53.700853 25692 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:12:53.702704 25692 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:12:53.714931 25703 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:12:53.715015 25699 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:12:53.715037 25698 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:12:53.716313 25692 server_base.cc:1034] running on GCE node
I20260807 18:12:53.717130 25692 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:12:53.718679 25692 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:12:53.719906 25692 hybrid_clock.cc:648] HybridClock initialized: now 1786126373719846 us; error 59 us; skew 500 ppm
I20260807 18:12:53.720299 25692 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 18:12:53.722887 25692 webserver.cc:460] Webserver started at http://127.18.154.62:44687/ using document root <none> and password file <none>
I20260807 18:12:53.723716 25692 fs_manager.cc:359] Metadata directory not provided
I20260807 18:12:53.723835 25692 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:12:53.724175 25692 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:12:53.727291 25692 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126328808105-19048-0/minicluster-data/master-0/data/instance:
uuid: "a2c1bf2caff94bc59213b9acede620e7"
format_stamp: "Formatted at 2026-08-07 18:12:53 on dist-test-slave-gkw7"
server_key: "6c0742f72519ffcae1d5b2459009a18b"
server_key_iv: "7da7cd09355bab8b0251d897f8a97d01"
server_key_version: "encryptionkey@0"
I20260807 18:12:53.728148 25692 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126328808105-19048-0/minicluster-data/master-0/wal/instance:
uuid: "a2c1bf2caff94bc59213b9acede620e7"
format_stamp: "Formatted at 2026-08-07 18:12:53 on dist-test-slave-gkw7"
server_key: "6c0742f72519ffcae1d5b2459009a18b"
server_key_iv: "7da7cd09355bab8b0251d897f8a97d01"
server_key_version: "encryptionkey@0"
I20260807 18:12:53.733667 25692 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260807 18:12:53.737282 25715 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:12:53.738888 25692 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:12:53.739090 25692 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126328808105-19048-0/minicluster-data/master-0/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126328808105-19048-0/minicluster-data/master-0/wal
uuid: "a2c1bf2caff94bc59213b9acede620e7"
format_stamp: "Formatted at 2026-08-07 18:12:53 on dist-test-slave-gkw7"
server_key: "6c0742f72519ffcae1d5b2459009a18b"
server_key_iv: "7da7cd09355bab8b0251d897f8a97d01"
server_key_version: "encryptionkey@0"
I20260807 18:12:53.739333 25692 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126328808105-19048-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126328808105-19048-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126328808105-19048-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:12:53.745448 25692 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:12:53.746340 25692 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:12:53.746665 25692 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:12:53.766237 25692 rpc_server.cc:307] RPC server started. Bound to: 127.18.154.62:34417
I20260807 18:12:53.766259 25781 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.154.62:34417 every 8 connection(s)
I20260807 18:12:53.767756 25692 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
I20260807 18:12:53.771581 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 25692
I20260807 18:12:53.771850 19048 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126328808105-19048-0/minicluster-data/master-0/wal/instance
I20260807 18:12:53.772087 25782 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:12:53.772725 19048 external_mini_cluster.cc:1428] Setting key 462d68dd0f33d5e0cbff986fba238ba1
I20260807 18:12:53.781584 25786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:53.774968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56418 (local address 127.18.154.62:34417)
0807 18:12:53.776095 (+ 1127us) server_negotiation.cc:206] Beginning negotiation
0807 18:12:53.776107 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 18:12:53.776156 (+ 49us) server_negotiation.cc:407] Connection header received
0807 18:12:53.777749 (+ 1593us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:12:53.777772 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:12:53.778217 (+ 445us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:12:53.778655 (+ 438us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:12:53.779845 (+ 1190us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:12:53.779854 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:12:53.779867 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:12:53.779881 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:12:53.779938 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:12:53.779952 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:12:53.780219 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0807 18:12:53.780359 (+ 140us) server_negotiation.cc:299] Negotiation successful
0807 18:12:53.780612 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":647,"thread_start_us":423,"threads_started":1}
I20260807 18:12:53.786091 25782 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a2c1bf2caff94bc59213b9acede620e7: Bootstrap starting.
I20260807 18:12:53.789369 25782 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a2c1bf2caff94bc59213b9acede620e7: Neither blocks nor log segments found. Creating new log.
I20260807 18:12:53.790679 25782 log.cc:824] T 00000000000000000000000000000000 P a2c1bf2caff94bc59213b9acede620e7: Log is configured to *not* fsync() on all Append() calls
I20260807 18:12:53.793596 25782 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a2c1bf2caff94bc59213b9acede620e7: No bootstrap required, opened a new log
I20260807 18:12:53.801456 25782 raft_consensus.cc:348] T 00000000000000000000000000000000 P a2c1bf2caff94bc59213b9acede620e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2c1bf2caff94bc59213b9acede620e7" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 34417 } }
I20260807 18:12:53.802086 25782 raft_consensus.cc:374] T 00000000000000000000000000000000 P a2c1bf2caff94bc59213b9acede620e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:12:53.802218 25782 raft_consensus.cc:729] T 00000000000000000000000000000000 P a2c1bf2caff94bc59213b9acede620e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2c1bf2caff94bc59213b9acede620e7, State: Initialized, Role: FOLLOWER
I20260807 18:12:53.803069 25782 consensus_queue.cc:260] T 00000000000000000000000000000000 P a2c1bf2caff94bc59213b9acede620e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2c1bf2caff94bc59213b9acede620e7" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 34417 } }
I20260807 18:12:53.803469 25782 raft_consensus.cc:388] T 00000000000000000000000000000000 P a2c1bf2caff94bc59213b9acede620e7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:12:53.803658 25782 raft_consensus.cc:482] T 00000000000000000000000000000000 P a2c1bf2caff94bc59213b9acede620e7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:12:53.803869 25782 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a2c1bf2caff94bc59213b9acede620e7 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:12:53.806424 25782 raft_consensus.cc:504] T 00000000000000000000000000000000 P a2c1bf2caff94bc59213b9acede620e7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2c1bf2caff94bc59213b9acede620e7" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 34417 } }
I20260807 18:12:53.807022 25782 leader_election.cc:302] T 00000000000000000000000000000000 P a2c1bf2caff94bc59213b9acede620e7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a2c1bf2caff94bc59213b9acede620e7; no voters:
I20260807 18:12:53.807664 25782 leader_election.cc:288] T 00000000000000000000000000000000 P a2c1bf2caff94bc59213b9acede620e7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:12:53.807847 25788 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a2c1bf2caff94bc59213b9acede620e7 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:12:53.808614 25788 raft_consensus.cc:686] T 00000000000000000000000000000000 P a2c1bf2caff94bc59213b9acede620e7 [term 1 LEADER]: Becoming Leader. State: Replica: a2c1bf2caff94bc59213b9acede620e7, State: Running, Role: LEADER
I20260807 18:12:53.809214 25788 consensus_queue.cc:237] T 00000000000000000000000000000000 P a2c1bf2caff94bc59213b9acede620e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2c1bf2caff94bc59213b9acede620e7" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 34417 } }
I20260807 18:12:53.809985 25782 sys_catalog.cc:564] T 00000000000000000000000000000000 P a2c1bf2caff94bc59213b9acede620e7 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:12:53.812227 25790 sys_catalog.cc:455] T 00000000000000000000000000000000 P a2c1bf2caff94bc59213b9acede620e7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a2c1bf2caff94bc59213b9acede620e7. Latest consensus state: current_term: 1 leader_uuid: "a2c1bf2caff94bc59213b9acede620e7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2c1bf2caff94bc59213b9acede620e7" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 34417 } } }
I20260807 18:12:53.812652 25790 sys_catalog.cc:458] T 00000000000000000000000000000000 P a2c1bf2caff94bc59213b9acede620e7 [sys.catalog]: This master's current role is: LEADER
I20260807 18:12:53.814307 25789 sys_catalog.cc:455] T 00000000000000000000000000000000 P a2c1bf2caff94bc59213b9acede620e7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a2c1bf2caff94bc59213b9acede620e7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2c1bf2caff94bc59213b9acede620e7" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 34417 } } }
I20260807 18:12:53.815088 25789 sys_catalog.cc:458] T 00000000000000000000000000000000 P a2c1bf2caff94bc59213b9acede620e7 [sys.catalog]: This master's current role is: LEADER
I20260807 18:12:53.815892 25797 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:12:53.822381 25797 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:12:53.832355 25797 catalog_manager.cc:1347] Generated new cluster ID: ea8bd35d99d747a9aa54749137fa02bf
I20260807 18:12:53.832526 25797 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:12:53.856858 25797 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:12:53.857468 25797 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:12:53.867419 25797 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a2c1bf2caff94bc59213b9acede620e7: Generated new TSK 0
I20260807 18:12:53.868531 25797 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:12:53.874436 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126328808105-19048-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126328808105-19048-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126328808105-19048-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126328808105-19048-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.154.1:0
--local_ip_for_outbound_sockets=127.18.154.1
--webserver_interface=127.18.154.1
--webserver_port=0
--tserver_master_addrs=127.18.154.62:34417
--builtin_ntp_servers=127.18.154.20:40235
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260807 18:12:54.228065 25809 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:12:54.228536 25809 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:12:54.228696 25809 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:12:54.235899 25809 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:12:54.236052 25809 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:12:54.236223 25809 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.154.1
I20260807 18:12:54.245519 25809 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.154.20:40235
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126328808105-19048-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126328808105-19048-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.154.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126328808105-19048-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.154.1
--webserver_port=0
--tserver_master_addrs=127.18.154.62:34417
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.154.1
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126328808105-19048-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:12:54.246920 25809 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:12:54.248790 25809 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:12:54.262730 25816 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:12:54.262820 25819 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:12:54.262780 25827 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:12:54.263813 25809 server_base.cc:1034] running on GCE node
I20260807 18:12:54.264485 25809 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:12:54.265579 25809 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:12:54.266820 25809 hybrid_clock.cc:648] HybridClock initialized: now 1786126374266754 us; error 67 us; skew 500 ppm
I20260807 18:12:54.267153 25809 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 18:12:54.273584 25809 webserver.cc:460] Webserver started at http://127.18.154.1:45659/ using document root <none> and password file <none>
I20260807 18:12:54.274369 25809 fs_manager.cc:359] Metadata directory not provided
I20260807 18:12:54.274511 25809 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:12:54.274885 25809 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:12:54.277928 25809 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126328808105-19048-0/minicluster-data/ts-0/data/instance:
uuid: "389a0c7a05d94b6abbba4db6e4b965a3"
format_stamp: "Formatted at 2026-08-07 18:12:54 on dist-test-slave-gkw7"
server_key: "c63f53a2cfc4c230bcf94cf055420991"
server_key_iv: "1ab9a0e1cb54db16033960f3553fca27"
server_key_version: "encryptionkey@0"
I20260807 18:12:54.278760 25809 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126328808105-19048-0/minicluster-data/ts-0/wal/instance:
uuid: "389a0c7a05d94b6abbba4db6e4b965a3"
format_stamp: "Formatted at 2026-08-07 18:12:54 on dist-test-slave-gkw7"
server_key: "c63f53a2cfc4c230bcf94cf055420991"
server_key_iv: "1ab9a0e1cb54db16033960f3553fca27"
server_key_version: "encryptionkey@0"
I20260807 18:12:54.284507 25809 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260807 18:12:54.288117 25835 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:12:54.289479 25809 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 18:12:54.289695 25809 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126328808105-19048-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126328808105-19048-0/minicluster-data/ts-0/wal
uuid: "389a0c7a05d94b6abbba4db6e4b965a3"
format_stamp: "Formatted at 2026-08-07 18:12:54 on dist-test-slave-gkw7"
server_key: "c63f53a2cfc4c230bcf94cf055420991"
server_key_iv: "1ab9a0e1cb54db16033960f3553fca27"
server_key_version: "encryptionkey@0"
I20260807 18:12:54.289934 25809 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126328808105-19048-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126328808105-19048-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126328808105-19048-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:12:54.295867 25809 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:12:54.296697 25809 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:12:54.296985 25809 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:12:54.297981 25809 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:12:54.299494 25809 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:12:54.299625 25809 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:12:54.299757 25809 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:12:54.299845 25809 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:12:54.334975 25809 rpc_server.cc:307] RPC server started. Bound to: 127.18.154.1:42857
I20260807 18:12:54.335070 25986 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.154.1:42857 every 8 connection(s)
I20260807 18:12:54.336362 25809 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126328808105-19048-0/minicluster-data/ts-0/data/info.pb
I20260807 18:12:54.340273 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 25809
I20260807 18:12:54.340507 19048 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126328808105-19048-0/minicluster-data/ts-0/wal/instance
I20260807 18:12:54.341315 19048 external_mini_cluster.cc:1428] Setting key ec157988e5eee81a96d366da7f6823bb
I20260807 18:12:54.344565 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126328808105-19048-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126328808105-19048-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126328808105-19048-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126328808105-19048-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.154.2:0
--local_ip_for_outbound_sockets=127.18.154.2
--webserver_interface=127.18.154.2
--webserver_port=0
--tserver_master_addrs=127.18.154.62:34417
--builtin_ntp_servers=127.18.154.20:40235
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260807 18:12:54.345811 25992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:54.339076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.1:40987 (local address 127.18.154.62:34417)
0807 18:12:54.339557 (+ 481us) server_negotiation.cc:206] Beginning negotiation
0807 18:12:54.339564 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:12:54.341111 (+ 1547us) server_negotiation.cc:407] Connection header received
0807 18:12:54.343176 (+ 2065us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:12:54.343183 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:12:54.343245 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:12:54.343370 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:12:54.345013 (+ 1643us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:12:54.345018 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:12:54.345021 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:12:54.345038 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:12:54.345084 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:12:54.345097 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:12:54.345235 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0807 18:12:54.345474 (+ 239us) server_negotiation.cc:299] Negotiation successful
0807 18:12:54.345571 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":340,"thread_start_us":212,"threads_started":1}
I20260807 18:12:54.347179 25993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:54.339683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:34417 (local address 127.18.154.1:40987)
0807 18:12:54.340942 (+ 1259us) negotiation.cc:107] Waiting for socket to connect
0807 18:12:54.340987 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 18:12:54.342786 (+ 1799us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:12:54.343664 (+ 878us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:12:54.343676 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:12:54.343815 (+ 139us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:12:54.344621 (+ 806us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 18:12:54.344625 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:12:54.344682 (+ 57us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 18:12:54.344826 (+ 144us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:12:54.345238 (+ 412us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:12:54.345252 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:12:54.345264 (+ 12us) client_negotiation.cc:769] Sending connection context
0807 18:12:54.345585 (+ 321us) client_negotiation.cc:240] Negotiation successful
0807 18:12:54.345879 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":715,"thread_start_us":443,"threads_started":1}
I20260807 18:12:54.348938 25988 heartbeater.cc:347] Connected to a master server at 127.18.154.62:34417
I20260807 18:12:54.349305 25988 heartbeater.cc:464] Registering TS with master...
I20260807 18:12:54.349992 25988 heartbeater.cc:511] Master 127.18.154.62:34417 requested a full tablet report, sending...
I20260807 18:12:54.352275 25739 ts_manager.cc:194] Registered new tserver with Master: 389a0c7a05d94b6abbba4db6e4b965a3 (127.18.154.1:42857)
W20260807 18:12:54.705725 25994 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:12:54.706058 25994 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:12:54.706154 25994 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:12:54.713979 25994 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:12:54.714095 25994 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:12:54.714263 25994 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.154.2
I20260807 18:12:54.723970 25994 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.154.20:40235
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126328808105-19048-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126328808105-19048-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.154.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126328808105-19048-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.154.2
--webserver_port=0
--tserver_master_addrs=127.18.154.62:34417
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.154.2
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126328808105-19048-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:12:54.725334 25994 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:12:54.727262 25994 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:12:54.741348 26001 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:12:54.741423 26002 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:12:54.741391 26004 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:12:54.742655 25994 server_base.cc:1034] running on GCE node
I20260807 18:12:54.743363 25994 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:12:54.744493 25994 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:12:54.745697 25994 hybrid_clock.cc:648] HybridClock initialized: now 1786126374745646 us; error 37 us; skew 500 ppm
I20260807 18:12:54.746052 25994 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 18:12:54.748517 25994 webserver.cc:460] Webserver started at http://127.18.154.2:43277/ using document root <none> and password file <none>
I20260807 18:12:54.749289 25994 fs_manager.cc:359] Metadata directory not provided
I20260807 18:12:54.749406 25994 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:12:54.749809 25994 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:12:54.752924 25994 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126328808105-19048-0/minicluster-data/ts-1/data/instance:
uuid: "3efae221ecc84934b4f8ee12cd84f21b"
format_stamp: "Formatted at 2026-08-07 18:12:54 on dist-test-slave-gkw7"
server_key: "acc7e68202414825a9b11015beb0db62"
server_key_iv: "67a032b86803340780bec85ba34a0f3b"
server_key_version: "encryptionkey@0"
I20260807 18:12:54.753845 25994 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126328808105-19048-0/minicluster-data/ts-1/wal/instance:
uuid: "3efae221ecc84934b4f8ee12cd84f21b"
format_stamp: "Formatted at 2026-08-07 18:12:54 on dist-test-slave-gkw7"
server_key: "acc7e68202414825a9b11015beb0db62"
server_key_iv: "67a032b86803340780bec85ba34a0f3b"
server_key_version: "encryptionkey@0"
I20260807 18:12:54.759466 25994 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260807 18:12:54.763428 26013 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:12:54.764921 25994 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 18:12:54.765110 25994 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126328808105-19048-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126328808105-19048-0/minicluster-data/ts-1/wal
uuid: "3efae221ecc84934b4f8ee12cd84f21b"
format_stamp: "Formatted at 2026-08-07 18:12:54 on dist-test-slave-gkw7"
server_key: "acc7e68202414825a9b11015beb0db62"
server_key_iv: "67a032b86803340780bec85ba34a0f3b"
server_key_version: "encryptionkey@0"
I20260807 18:12:54.765374 25994 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126328808105-19048-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126328808105-19048-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126328808105-19048-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:12:54.771453 25994 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:12:54.772285 25994 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:12:54.772579 25994 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:12:54.773582 25994 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:12:54.775172 25994 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:12:54.775269 25994 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:12:54.775425 25994 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:12:54.775516 25994 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:12:54.811396 25994 rpc_server.cc:307] RPC server started. Bound to: 127.18.154.2:33399
I20260807 18:12:54.811455 26154 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.154.2:33399 every 8 connection(s)
I20260807 18:12:54.812912 25994 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126328808105-19048-0/minicluster-data/ts-1/data/info.pb
I20260807 18:12:54.822220 25992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:54.816126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.2:43509 (local address 127.18.154.62:34417)
0807 18:12:54.816376 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0807 18:12:54.816388 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 18:12:54.817948 (+ 1560us) server_negotiation.cc:407] Connection header received
0807 18:12:54.819718 (+ 1770us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:12:54.819723 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:12:54.819782 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:12:54.819880 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:12:54.821269 (+ 1389us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:12:54.821274 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:12:54.821277 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:12:54.821297 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:12:54.821334 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:12:54.821346 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:12:54.821451 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0807 18:12:54.821726 (+ 275us) server_negotiation.cc:299] Negotiation successful
0807 18:12:54.821799 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260807 18:12:54.822311 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 25994
I20260807 18:12:54.822506 19048 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126328808105-19048-0/minicluster-data/ts-1/wal/instance
I20260807 18:12:54.823094 19048 external_mini_cluster.cc:1428] Setting key 86edcca8286b620f839b3a3f949af148
I20260807 18:12:54.823311 26159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:54.816725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:34417 (local address 127.18.154.2:43509)
0807 18:12:54.817781 (+ 1056us) negotiation.cc:107] Waiting for socket to connect
0807 18:12:54.817821 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 18:12:54.819415 (+ 1594us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:12:54.820113 (+ 698us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:12:54.820122 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:12:54.820241 (+ 119us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:12:54.820939 (+ 698us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 18:12:54.820941 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:12:54.820984 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 18:12:54.821079 (+ 95us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:12:54.821454 (+ 375us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:12:54.821463 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:12:54.821469 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 18:12:54.821796 (+ 327us) client_negotiation.cc:240] Negotiation successful
0807 18:12:54.822033 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":492,"thread_start_us":279,"threads_started":1}
I20260807 18:12:54.824983 26157 heartbeater.cc:347] Connected to a master server at 127.18.154.62:34417
I20260807 18:12:54.825358 26157 heartbeater.cc:464] Registering TS with master...
I20260807 18:12:54.826118 26157 heartbeater.cc:511] Master 127.18.154.62:34417 requested a full tablet report, sending...
I20260807 18:12:54.826653 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126328808105-19048-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126328808105-19048-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126328808105-19048-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126328808105-19048-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.154.3:0
--local_ip_for_outbound_sockets=127.18.154.3
--webserver_interface=127.18.154.3
--webserver_port=0
--tserver_master_addrs=127.18.154.62:34417
--builtin_ntp_servers=127.18.154.20:40235
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260807 18:12:54.827732 25738 ts_manager.cc:194] Registered new tserver with Master: 3efae221ecc84934b4f8ee12cd84f21b (127.18.154.2:33399)
W20260807 18:12:55.176142 26161 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:12:55.176440 26161 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:12:55.176602 26161 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:12:55.184042 26161 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:12:55.184150 26161 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:12:55.184334 26161 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.154.3
I20260807 18:12:55.193776 26161 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.154.20:40235
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126328808105-19048-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126328808105-19048-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.154.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126328808105-19048-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.154.3
--webserver_port=0
--tserver_master_addrs=127.18.154.62:34417
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.154.3
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126328808105-19048-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:12:55.195258 26161 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:12:55.197191 26161 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:12:55.211403 26170 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:12:55.211716 26169 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:12:55.211655 26172 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:12:55.212324 26161 server_base.cc:1034] running on GCE node
I20260807 18:12:55.213002 26161 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:12:55.214078 26161 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:12:55.215330 26161 hybrid_clock.cc:648] HybridClock initialized: now 1786126375215259 us; error 61 us; skew 500 ppm
I20260807 18:12:55.215693 26161 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 18:12:55.218245 26161 webserver.cc:460] Webserver started at http://127.18.154.3:39471/ using document root <none> and password file <none>
I20260807 18:12:55.218993 26161 fs_manager.cc:359] Metadata directory not provided
I20260807 18:12:55.219133 26161 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:12:55.219486 26161 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:12:55.222579 26161 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126328808105-19048-0/minicluster-data/ts-2/data/instance:
uuid: "1e026338e21940b1953ca8618702d3a7"
format_stamp: "Formatted at 2026-08-07 18:12:55 on dist-test-slave-gkw7"
server_key: "47719ec0310b48bd9d816150ac71786f"
server_key_iv: "2a9cfc08e70f02b659dee0d2449c1570"
server_key_version: "encryptionkey@0"
I20260807 18:12:55.223421 26161 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126328808105-19048-0/minicluster-data/ts-2/wal/instance:
uuid: "1e026338e21940b1953ca8618702d3a7"
format_stamp: "Formatted at 2026-08-07 18:12:55 on dist-test-slave-gkw7"
server_key: "47719ec0310b48bd9d816150ac71786f"
server_key_iv: "2a9cfc08e70f02b659dee0d2449c1570"
server_key_version: "encryptionkey@0"
I20260807 18:12:55.228848 26161 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 18:12:55.232437 26178 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:12:55.233927 26161 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 18:12:55.234109 26161 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126328808105-19048-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126328808105-19048-0/minicluster-data/ts-2/wal
uuid: "1e026338e21940b1953ca8618702d3a7"
format_stamp: "Formatted at 2026-08-07 18:12:55 on dist-test-slave-gkw7"
server_key: "47719ec0310b48bd9d816150ac71786f"
server_key_iv: "2a9cfc08e70f02b659dee0d2449c1570"
server_key_version: "encryptionkey@0"
I20260807 18:12:55.234342 26161 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126328808105-19048-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126328808105-19048-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126328808105-19048-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:12:55.240114 26161 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:12:55.241007 26161 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:12:55.241290 26161 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:12:55.242365 26161 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:12:55.243925 26161 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:12:55.244045 26161 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:12:55.244180 26161 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:12:55.244275 26161 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:12:55.284427 26161 rpc_server.cc:307] RPC server started. Bound to: 127.18.154.3:38989
I20260807 18:12:55.284478 26325 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.154.3:38989 every 8 connection(s)
I20260807 18:12:55.285946 26161 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126328808105-19048-0/minicluster-data/ts-2/data/info.pb
I20260807 18:12:55.292227 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 26161
I20260807 18:12:55.292456 19048 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126328808105-19048-0/minicluster-data/ts-2/wal/instance
I20260807 18:12:55.293113 19048 external_mini_cluster.cc:1428] Setting key 6d5bb4ea1b216297b7ab4b7a865b5245
I20260807 18:12:55.296701 25992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:55.288732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.3:42327 (local address 127.18.154.62:34417)
0807 18:12:55.289779 (+ 1047us) server_negotiation.cc:206] Beginning negotiation
0807 18:12:55.289788 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:12:55.290856 (+ 1068us) server_negotiation.cc:407] Connection header received
0807 18:12:55.293638 (+ 2782us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:12:55.293643 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:12:55.293703 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:12:55.293884 (+ 181us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:12:55.295836 (+ 1952us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:12:55.295842 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:12:55.295846 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:12:55.295865 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:12:55.295903 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:12:55.295918 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:12:55.296042 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0807 18:12:55.296284 (+ 242us) server_negotiation.cc:299] Negotiation successful
0807 18:12:55.296350 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":859}
I20260807 18:12:55.298013 26329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:55.289349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:34417 (local address 127.18.154.3:42327)
0807 18:12:55.290675 (+ 1326us) negotiation.cc:107] Waiting for socket to connect
0807 18:12:55.290728 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0807 18:12:55.293229 (+ 2501us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:12:55.294279 (+ 1050us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:12:55.294293 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:12:55.294443 (+ 150us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:12:55.295359 (+ 916us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 18:12:55.295362 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:12:55.295454 (+ 92us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 18:12:55.295624 (+ 170us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:12:55.296045 (+ 421us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:12:55.296055 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:12:55.296062 (+ 7us) client_negotiation.cc:769] Sending connection context
0807 18:12:55.296352 (+ 290us) client_negotiation.cc:240] Negotiation successful
0807 18:12:55.296701 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":790,"thread_start_us":525,"threads_started":1}
I20260807 18:12:55.299343 26327 heartbeater.cc:347] Connected to a master server at 127.18.154.62:34417
I20260807 18:12:55.299659 26327 heartbeater.cc:464] Registering TS with master...
I20260807 18:12:55.300290 26327 heartbeater.cc:511] Master 127.18.154.62:34417 requested a full tablet report, sending...
I20260807 18:12:55.301795 25738 ts_manager.cc:194] Registered new tserver with Master: 1e026338e21940b1953ca8618702d3a7 (127.18.154.3:38989)
I20260807 18:12:55.309653 19048 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 18:12:55.317726 25992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:55.314843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56422 (local address 127.18.154.62:34417)
0807 18:12:55.315063 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0807 18:12:55.315071 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:12:55.315665 (+ 594us) server_negotiation.cc:407] Connection header received
0807 18:12:55.315878 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:12:55.315884 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:12:55.315952 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:12:55.316059 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:12:55.316990 (+ 931us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:12:55.316996 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:12:55.317000 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:12:55.317019 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:12:55.317061 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:12:55.317073 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:12:55.317203 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0807 18:12:55.317344 (+ 141us) server_negotiation.cc:299] Negotiation successful
0807 18:12:55.317410 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260807 18:12:55.322324 25738 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56422:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 18:12:55.329504 25738 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:12:55.355422 26345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:55.344452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42402 (local address 127.18.154.1:42857)
0807 18:12:55.349911 (+ 5459us) server_negotiation.cc:206] Beginning negotiation
0807 18:12:55.349926 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 18:12:55.350865 (+ 939us) server_negotiation.cc:407] Connection header received
0807 18:12:55.351082 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:12:55.351091 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:12:55.351325 (+ 234us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:12:55.351483 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:12:55.354069 (+ 2586us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:12:55.354083 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:12:55.354106 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:12:55.354142 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:12:55.354220 (+ 78us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:12:55.354243 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:12:55.354440 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0807 18:12:55.354753 (+ 313us) server_negotiation.cc:299] Negotiation successful
0807 18:12:55.354889 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5194,"thread_start_us":4955,"threads_started":1}
I20260807 18:12:55.355554 26344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:55.343224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.3:38989 (local address 127.0.0.1:42654)
0807 18:12:55.344747 (+ 1523us) negotiation.cc:107] Waiting for socket to connect
0807 18:12:55.344800 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0807 18:12:55.344991 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:12:55.351329 (+ 6338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:12:55.351340 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:12:55.351472 (+ 132us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:12:55.352541 (+ 1069us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 18:12:55.352547 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:12:55.352592 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 18:12:55.352721 (+ 129us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:12:55.354467 (+ 1746us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:12:55.354486 (+ 19us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:12:55.354500 (+ 14us) client_negotiation.cc:769] Sending connection context
0807 18:12:55.354739 (+ 239us) client_negotiation.cc:240] Negotiation successful
0807 18:12:55.354867 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1171,"thread_start_us":7244,"threads_started":1}
I20260807 18:12:55.355769 25988 heartbeater.cc:503] Master 127.18.154.62:34417 was elected leader, sending a full tablet report...
I20260807 18:12:55.357384 26342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:55.343642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.1:42857 (local address 127.0.0.1:42402)
0807 18:12:55.350727 (+ 7085us) negotiation.cc:107] Waiting for socket to connect
0807 18:12:55.350746 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 18:12:55.350896 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:12:55.352876 (+ 1980us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:12:55.352885 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:12:55.352919 (+ 34us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:12:55.353757 (+ 838us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 18:12:55.353763 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:12:55.353786 (+ 23us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 18:12:55.353846 (+ 60us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:12:55.354505 (+ 659us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:12:55.354512 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:12:55.354521 (+ 9us) client_negotiation.cc:769] Sending connection context
0807 18:12:55.356637 (+ 2116us) client_negotiation.cc:240] Negotiation successful
0807 18:12:55.356727 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6944,"thread_start_us":1051,"threads_started":1}
I20260807 18:12:55.358906 26344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:55.351248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.2:33399 (local address 127.0.0.1:51720)
0807 18:12:55.355791 (+ 4543us) negotiation.cc:107] Waiting for socket to connect
0807 18:12:55.355814 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 18:12:55.355899 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:12:55.356645 (+ 746us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:12:55.356649 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:12:55.356695 (+ 46us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:12:55.357290 (+ 595us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 18:12:55.357292 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:12:55.357308 (+ 16us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 18:12:55.357354 (+ 46us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:12:55.358546 (+ 1192us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:12:55.358551 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:12:55.358556 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 18:12:55.358635 (+ 79us) client_negotiation.cc:240] Negotiation successful
0807 18:12:55.358685 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4424,"threads_started":1}
I20260807 18:12:55.359122 26343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:55.343390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42654 (local address 127.18.154.3:38989)
0807 18:12:55.350625 (+ 7235us) server_negotiation.cc:206] Beginning negotiation
0807 18:12:55.350633 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:12:55.350675 (+ 42us) server_negotiation.cc:407] Connection header received
0807 18:12:55.350765 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:12:55.350777 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:12:55.351083 (+ 306us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:12:55.352139 (+ 1056us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:12:55.354112 (+ 1973us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:12:55.354125 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:12:55.354145 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:12:55.354170 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:12:55.354249 (+ 79us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:12:55.354277 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:12:55.358401 (+ 4124us) server_negotiation.cc:1035] Waiting for connection context
0807 18:12:55.358522 (+ 121us) server_negotiation.cc:299] Negotiation successful
0807 18:12:55.358657 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7031,"thread_start_us":6795,"threads_started":1}
I20260807 18:12:55.359616 26348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:55.351335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51720 (local address 127.18.154.2:33399)
0807 18:12:55.356116 (+ 4781us) server_negotiation.cc:206] Beginning negotiation
0807 18:12:55.356124 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:12:55.356153 (+ 29us) server_negotiation.cc:407] Connection header received
0807 18:12:55.356220 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:12:55.356228 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:12:55.356454 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:12:55.356987 (+ 533us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:12:55.358291 (+ 1304us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:12:55.358299 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:12:55.358313 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:12:55.358341 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:12:55.358392 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:12:55.358410 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:12:55.359141 (+ 731us) server_negotiation.cc:1035] Waiting for connection context
0807 18:12:55.359274 (+ 133us) server_negotiation.cc:299] Negotiation successful
0807 18:12:55.359368 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4598,"thread_start_us":4467,"threads_started":1}
I20260807 18:12:55.362344 25902 tablet_service.cc:1467] Processing CreateTablet for tablet 93602adfc5b841daac8dcd38836db5ed (DEFAULT_TABLE table=test-table [id=c06aa6a9eb4143faacb7847fd8b47337]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:12:55.363734 25902 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 93602adfc5b841daac8dcd38836db5ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:12:55.364114 26072 tablet_service.cc:1467] Processing CreateTablet for tablet 93602adfc5b841daac8dcd38836db5ed (DEFAULT_TABLE table=test-table [id=c06aa6a9eb4143faacb7847fd8b47337]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:12:55.364115 26243 tablet_service.cc:1467] Processing CreateTablet for tablet 93602adfc5b841daac8dcd38836db5ed (DEFAULT_TABLE table=test-table [id=c06aa6a9eb4143faacb7847fd8b47337]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:12:55.365518 26243 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 93602adfc5b841daac8dcd38836db5ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:12:55.365516 26072 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 93602adfc5b841daac8dcd38836db5ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:12:55.377332 26349 tablet_bootstrap.cc:492] T 93602adfc5b841daac8dcd38836db5ed P 389a0c7a05d94b6abbba4db6e4b965a3: Bootstrap starting.
I20260807 18:12:55.380966 26350 tablet_bootstrap.cc:492] T 93602adfc5b841daac8dcd38836db5ed P 1e026338e21940b1953ca8618702d3a7: Bootstrap starting.
I20260807 18:12:55.382606 26349 tablet_bootstrap.cc:654] T 93602adfc5b841daac8dcd38836db5ed P 389a0c7a05d94b6abbba4db6e4b965a3: Neither blocks nor log segments found. Creating new log.
I20260807 18:12:55.384130 26349 log.cc:824] T 93602adfc5b841daac8dcd38836db5ed P 389a0c7a05d94b6abbba4db6e4b965a3: Log is configured to *not* fsync() on all Append() calls
I20260807 18:12:55.386320 26351 tablet_bootstrap.cc:492] T 93602adfc5b841daac8dcd38836db5ed P 3efae221ecc84934b4f8ee12cd84f21b: Bootstrap starting.
I20260807 18:12:55.386677 26350 tablet_bootstrap.cc:654] T 93602adfc5b841daac8dcd38836db5ed P 1e026338e21940b1953ca8618702d3a7: Neither blocks nor log segments found. Creating new log.
I20260807 18:12:55.388387 26350 log.cc:824] T 93602adfc5b841daac8dcd38836db5ed P 1e026338e21940b1953ca8618702d3a7: Log is configured to *not* fsync() on all Append() calls
I20260807 18:12:55.389633 26351 tablet_bootstrap.cc:654] T 93602adfc5b841daac8dcd38836db5ed P 3efae221ecc84934b4f8ee12cd84f21b: Neither blocks nor log segments found. Creating new log.
I20260807 18:12:55.390906 26351 log.cc:824] T 93602adfc5b841daac8dcd38836db5ed P 3efae221ecc84934b4f8ee12cd84f21b: Log is configured to *not* fsync() on all Append() calls
I20260807 18:12:55.392149 26350 tablet_bootstrap.cc:492] T 93602adfc5b841daac8dcd38836db5ed P 1e026338e21940b1953ca8618702d3a7: No bootstrap required, opened a new log
I20260807 18:12:55.392482 26350 ts_tablet_manager.cc:1397] T 93602adfc5b841daac8dcd38836db5ed P 1e026338e21940b1953ca8618702d3a7: Time spent bootstrapping tablet: real 0.012s user 0.008s sys 0.000s
I20260807 18:12:55.392976 26349 tablet_bootstrap.cc:492] T 93602adfc5b841daac8dcd38836db5ed P 389a0c7a05d94b6abbba4db6e4b965a3: No bootstrap required, opened a new log
I20260807 18:12:55.393291 26349 ts_tablet_manager.cc:1397] T 93602adfc5b841daac8dcd38836db5ed P 389a0c7a05d94b6abbba4db6e4b965a3: Time spent bootstrapping tablet: real 0.016s user 0.003s sys 0.011s
I20260807 18:12:55.394856 26351 tablet_bootstrap.cc:492] T 93602adfc5b841daac8dcd38836db5ed P 3efae221ecc84934b4f8ee12cd84f21b: No bootstrap required, opened a new log
I20260807 18:12:55.395164 26351 ts_tablet_manager.cc:1397] T 93602adfc5b841daac8dcd38836db5ed P 3efae221ecc84934b4f8ee12cd84f21b: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260807 18:12:55.401589 26349 raft_consensus.cc:348] T 93602adfc5b841daac8dcd38836db5ed P 389a0c7a05d94b6abbba4db6e4b965a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e026338e21940b1953ca8618702d3a7" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 38989 } } peers { permanent_uuid: "389a0c7a05d94b6abbba4db6e4b965a3" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 42857 } } peers { permanent_uuid: "3efae221ecc84934b4f8ee12cd84f21b" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 33399 } }
I20260807 18:12:55.402148 26349 raft_consensus.cc:374] T 93602adfc5b841daac8dcd38836db5ed P 389a0c7a05d94b6abbba4db6e4b965a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:12:55.402266 26349 raft_consensus.cc:729] T 93602adfc5b841daac8dcd38836db5ed P 389a0c7a05d94b6abbba4db6e4b965a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 389a0c7a05d94b6abbba4db6e4b965a3, State: Initialized, Role: FOLLOWER
I20260807 18:12:55.402911 26349 consensus_queue.cc:260] T 93602adfc5b841daac8dcd38836db5ed P 389a0c7a05d94b6abbba4db6e4b965a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e026338e21940b1953ca8618702d3a7" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 38989 } } peers { permanent_uuid: "389a0c7a05d94b6abbba4db6e4b965a3" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 42857 } } peers { permanent_uuid: "3efae221ecc84934b4f8ee12cd84f21b" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 33399 } }
I20260807 18:12:55.404819 26349 ts_tablet_manager.cc:1428] T 93602adfc5b841daac8dcd38836db5ed P 389a0c7a05d94b6abbba4db6e4b965a3: Time spent starting tablet: real 0.011s user 0.007s sys 0.003s
I20260807 18:12:55.404578 26350 raft_consensus.cc:348] T 93602adfc5b841daac8dcd38836db5ed P 1e026338e21940b1953ca8618702d3a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e026338e21940b1953ca8618702d3a7" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 38989 } } peers { permanent_uuid: "389a0c7a05d94b6abbba4db6e4b965a3" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 42857 } } peers { permanent_uuid: "3efae221ecc84934b4f8ee12cd84f21b" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 33399 } }
I20260807 18:12:55.405105 26350 raft_consensus.cc:374] T 93602adfc5b841daac8dcd38836db5ed P 1e026338e21940b1953ca8618702d3a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:12:55.405244 26350 raft_consensus.cc:729] T 93602adfc5b841daac8dcd38836db5ed P 1e026338e21940b1953ca8618702d3a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e026338e21940b1953ca8618702d3a7, State: Initialized, Role: FOLLOWER
I20260807 18:12:55.406024 26350 consensus_queue.cc:260] T 93602adfc5b841daac8dcd38836db5ed P 1e026338e21940b1953ca8618702d3a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e026338e21940b1953ca8618702d3a7" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 38989 } } peers { permanent_uuid: "389a0c7a05d94b6abbba4db6e4b965a3" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 42857 } } peers { permanent_uuid: "3efae221ecc84934b4f8ee12cd84f21b" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 33399 } }
I20260807 18:12:55.407261 26327 heartbeater.cc:503] Master 127.18.154.62:34417 was elected leader, sending a full tablet report...
I20260807 18:12:55.407963 26350 ts_tablet_manager.cc:1428] T 93602adfc5b841daac8dcd38836db5ed P 1e026338e21940b1953ca8618702d3a7: Time spent starting tablet: real 0.015s user 0.012s sys 0.004s
I20260807 18:12:55.408193 26351 raft_consensus.cc:348] T 93602adfc5b841daac8dcd38836db5ed P 3efae221ecc84934b4f8ee12cd84f21b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e026338e21940b1953ca8618702d3a7" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 38989 } } peers { permanent_uuid: "389a0c7a05d94b6abbba4db6e4b965a3" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 42857 } } peers { permanent_uuid: "3efae221ecc84934b4f8ee12cd84f21b" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 33399 } }
I20260807 18:12:55.408695 26351 raft_consensus.cc:374] T 93602adfc5b841daac8dcd38836db5ed P 3efae221ecc84934b4f8ee12cd84f21b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:12:55.408804 26351 raft_consensus.cc:729] T 93602adfc5b841daac8dcd38836db5ed P 3efae221ecc84934b4f8ee12cd84f21b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3efae221ecc84934b4f8ee12cd84f21b, State: Initialized, Role: FOLLOWER
I20260807 18:12:55.409502 26351 consensus_queue.cc:260] T 93602adfc5b841daac8dcd38836db5ed P 3efae221ecc84934b4f8ee12cd84f21b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e026338e21940b1953ca8618702d3a7" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 38989 } } peers { permanent_uuid: "389a0c7a05d94b6abbba4db6e4b965a3" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 42857 } } peers { permanent_uuid: "3efae221ecc84934b4f8ee12cd84f21b" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 33399 } }
I20260807 18:12:55.411244 26157 heartbeater.cc:503] Master 127.18.154.62:34417 was elected leader, sending a full tablet report...
I20260807 18:12:55.411720 26351 ts_tablet_manager.cc:1428] T 93602adfc5b841daac8dcd38836db5ed P 3efae221ecc84934b4f8ee12cd84f21b: Time spent starting tablet: real 0.016s user 0.016s sys 0.000s
I20260807 18:12:55.498689 26357 raft_consensus.cc:482] T 93602adfc5b841daac8dcd38836db5ed P 389a0c7a05d94b6abbba4db6e4b965a3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:12:55.499009 26357 raft_consensus.cc:504] T 93602adfc5b841daac8dcd38836db5ed P 389a0c7a05d94b6abbba4db6e4b965a3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e026338e21940b1953ca8618702d3a7" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 38989 } } peers { permanent_uuid: "389a0c7a05d94b6abbba4db6e4b965a3" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 42857 } } peers { permanent_uuid: "3efae221ecc84934b4f8ee12cd84f21b" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 33399 } }
I20260807 18:12:55.500864 26357 leader_election.cc:288] T 93602adfc5b841daac8dcd38836db5ed P 389a0c7a05d94b6abbba4db6e4b965a3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1e026338e21940b1953ca8618702d3a7 (127.18.154.3:38989), 3efae221ecc84934b4f8ee12cd84f21b (127.18.154.2:33399)
I20260807 18:12:55.504500 26343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:55.501489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.1:51285 (local address 127.18.154.3:38989)
0807 18:12:55.501756 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0807 18:12:55.501762 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:12:55.502032 (+ 270us) server_negotiation.cc:407] Connection header received
0807 18:12:55.502349 (+ 317us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:12:55.502358 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:12:55.502435 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:12:55.502516 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:12:55.503838 (+ 1322us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:12:55.503845 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:12:55.503850 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:12:55.503869 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:12:55.503910 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:12:55.503925 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:12:55.504056 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0807 18:12:55.504154 (+ 98us) server_negotiation.cc:299] Negotiation successful
0807 18:12:55.504218 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103}
I20260807 18:12:55.504496 26348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:55.502255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.1:59893 (local address 127.18.154.2:33399)
0807 18:12:55.502459 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0807 18:12:55.502465 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:12:55.502790 (+ 325us) server_negotiation.cc:407] Connection header received
0807 18:12:55.502914 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:12:55.502918 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:12:55.502970 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:12:55.503054 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:12:55.503838 (+ 784us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:12:55.503845 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:12:55.503850 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:12:55.503870 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:12:55.503909 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:12:55.503925 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:12:55.504047 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0807 18:12:55.504152 (+ 105us) server_negotiation.cc:299] Negotiation successful
0807 18:12:55.504238 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260807 18:12:55.504834 26363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:55.501939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.2:33399 (local address 127.18.154.1:59893)
0807 18:12:55.502685 (+ 746us) negotiation.cc:107] Waiting for socket to connect
0807 18:12:55.502703 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 18:12:55.502800 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:12:55.503137 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:12:55.503144 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:12:55.503172 (+ 28us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:12:55.503646 (+ 474us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 18:12:55.503648 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:12:55.503659 (+ 11us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 18:12:55.503686 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:12:55.504036 (+ 350us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:12:55.504041 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:12:55.504046 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 18:12:55.504222 (+ 176us) client_negotiation.cc:240] Negotiation successful
0807 18:12:55.504285 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":618,"mutex_wait_us":20,"thread_start_us":416,"threads_started":1}
I20260807 18:12:55.505167 26362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:55.501105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.3:38989 (local address 127.18.154.1:51285)
0807 18:12:55.501913 (+ 808us) negotiation.cc:107] Waiting for socket to connect
0807 18:12:55.501935 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 18:12:55.502145 (+ 210us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:12:55.502769 (+ 624us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:12:55.502773 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:12:55.502796 (+ 23us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:12:55.503572 (+ 776us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 18:12:55.503579 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:12:55.503602 (+ 23us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 18:12:55.503669 (+ 67us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:12:55.504036 (+ 367us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:12:55.504041 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:12:55.504046 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 18:12:55.504195 (+ 149us) client_negotiation.cc:240] Negotiation successful
0807 18:12:55.504273 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":611,"thread_start_us":330,"threads_started":1}
I20260807 18:12:55.506152 26097 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93602adfc5b841daac8dcd38836db5ed" candidate_uuid: "389a0c7a05d94b6abbba4db6e4b965a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3efae221ecc84934b4f8ee12cd84f21b" is_pre_election: true
I20260807 18:12:55.506179 26269 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93602adfc5b841daac8dcd38836db5ed" candidate_uuid: "389a0c7a05d94b6abbba4db6e4b965a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e026338e21940b1953ca8618702d3a7" is_pre_election: true
I20260807 18:12:55.506803 26269 raft_consensus.cc:2454] T 93602adfc5b841daac8dcd38836db5ed P 1e026338e21940b1953ca8618702d3a7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 389a0c7a05d94b6abbba4db6e4b965a3 in term 0.
I20260807 18:12:55.506803 26097 raft_consensus.cc:2454] T 93602adfc5b841daac8dcd38836db5ed P 3efae221ecc84934b4f8ee12cd84f21b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 389a0c7a05d94b6abbba4db6e4b965a3 in term 0.
I20260807 18:12:55.507524 25840 leader_election.cc:302] T 93602adfc5b841daac8dcd38836db5ed P 389a0c7a05d94b6abbba4db6e4b965a3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1e026338e21940b1953ca8618702d3a7, 389a0c7a05d94b6abbba4db6e4b965a3; no voters:
I20260807 18:12:55.507874 26357 raft_consensus.cc:2781] T 93602adfc5b841daac8dcd38836db5ed P 389a0c7a05d94b6abbba4db6e4b965a3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:12:55.508046 26357 raft_consensus.cc:482] T 93602adfc5b841daac8dcd38836db5ed P 389a0c7a05d94b6abbba4db6e4b965a3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:12:55.508198 26357 raft_consensus.cc:3037] T 93602adfc5b841daac8dcd38836db5ed P 389a0c7a05d94b6abbba4db6e4b965a3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:12:55.510766 26357 raft_consensus.cc:504] T 93602adfc5b841daac8dcd38836db5ed P 389a0c7a05d94b6abbba4db6e4b965a3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e026338e21940b1953ca8618702d3a7" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 38989 } } peers { permanent_uuid: "389a0c7a05d94b6abbba4db6e4b965a3" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 42857 } } peers { permanent_uuid: "3efae221ecc84934b4f8ee12cd84f21b" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 33399 } }
I20260807 18:12:55.511479 26357 leader_election.cc:288] T 93602adfc5b841daac8dcd38836db5ed P 389a0c7a05d94b6abbba4db6e4b965a3 [CANDIDATE]: Term 1 election: Requested vote from peers 1e026338e21940b1953ca8618702d3a7 (127.18.154.3:38989), 3efae221ecc84934b4f8ee12cd84f21b (127.18.154.2:33399)
I20260807 18:12:55.511956 26269 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93602adfc5b841daac8dcd38836db5ed" candidate_uuid: "389a0c7a05d94b6abbba4db6e4b965a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e026338e21940b1953ca8618702d3a7"
I20260807 18:12:55.512200 26269 raft_consensus.cc:3037] T 93602adfc5b841daac8dcd38836db5ed P 1e026338e21940b1953ca8618702d3a7 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:12:55.512231 26097 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93602adfc5b841daac8dcd38836db5ed" candidate_uuid: "389a0c7a05d94b6abbba4db6e4b965a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3efae221ecc84934b4f8ee12cd84f21b"
I20260807 18:12:55.512460 26097 raft_consensus.cc:3037] T 93602adfc5b841daac8dcd38836db5ed P 3efae221ecc84934b4f8ee12cd84f21b [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:12:55.515779 26097 raft_consensus.cc:2454] T 93602adfc5b841daac8dcd38836db5ed P 3efae221ecc84934b4f8ee12cd84f21b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 389a0c7a05d94b6abbba4db6e4b965a3 in term 1.
I20260807 18:12:55.515765 26269 raft_consensus.cc:2454] T 93602adfc5b841daac8dcd38836db5ed P 1e026338e21940b1953ca8618702d3a7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 389a0c7a05d94b6abbba4db6e4b965a3 in term 1.
I20260807 18:12:55.516270 25840 leader_election.cc:302] T 93602adfc5b841daac8dcd38836db5ed P 389a0c7a05d94b6abbba4db6e4b965a3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 389a0c7a05d94b6abbba4db6e4b965a3, 3efae221ecc84934b4f8ee12cd84f21b; no voters:
I20260807 18:12:55.516600 26357 raft_consensus.cc:2781] T 93602adfc5b841daac8dcd38836db5ed P 389a0c7a05d94b6abbba4db6e4b965a3 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:12:55.517185 26357 raft_consensus.cc:686] T 93602adfc5b841daac8dcd38836db5ed P 389a0c7a05d94b6abbba4db6e4b965a3 [term 1 LEADER]: Becoming Leader. State: Replica: 389a0c7a05d94b6abbba4db6e4b965a3, State: Running, Role: LEADER
I20260807 18:12:55.518114 26357 consensus_queue.cc:237] T 93602adfc5b841daac8dcd38836db5ed P 389a0c7a05d94b6abbba4db6e4b965a3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e026338e21940b1953ca8618702d3a7" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 38989 } } peers { permanent_uuid: "389a0c7a05d94b6abbba4db6e4b965a3" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 42857 } } peers { permanent_uuid: "3efae221ecc84934b4f8ee12cd84f21b" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 33399 } }
I20260807 18:12:55.522923 25739 catalog_manager.cc:5495] T 93602adfc5b841daac8dcd38836db5ed P 389a0c7a05d94b6abbba4db6e4b965a3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 389a0c7a05d94b6abbba4db6e4b965a3 (127.18.154.1). New cstate: current_term: 1 leader_uuid: "389a0c7a05d94b6abbba4db6e4b965a3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e026338e21940b1953ca8618702d3a7" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 38989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "389a0c7a05d94b6abbba4db6e4b965a3" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 42857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3efae221ecc84934b4f8ee12cd84f21b" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 33399 } health_report { overall_health: UNKNOWN } } }
W20260807 18:12:55.537853 26328 tablet.cc:2369] T 93602adfc5b841daac8dcd38836db5ed P 1e026338e21940b1953ca8618702d3a7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:12:55.564535 26345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:55.562508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42410 (local address 127.18.154.1:42857)
0807 18:12:55.562682 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0807 18:12:55.562687 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:12:55.562704 (+ 17us) server_negotiation.cc:407] Connection header received
0807 18:12:55.562870 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:12:55.562874 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:12:55.562962 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:12:55.563050 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:12:55.563996 (+ 946us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:12:55.564000 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:12:55.564003 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:12:55.564021 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:12:55.564054 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:12:55.564065 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:12:55.564166 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0807 18:12:55.564292 (+ 126us) server_negotiation.cc:299] Negotiation successful
0807 18:12:55.564342 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
W20260807 18:12:55.565286 26158 tablet.cc:2369] T 93602adfc5b841daac8dcd38836db5ed P 3efae221ecc84934b4f8ee12cd84f21b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:12:55.573474 26269 raft_consensus.cc:1261] T 93602adfc5b841daac8dcd38836db5ed P 1e026338e21940b1953ca8618702d3a7 [term 1 FOLLOWER]: Refusing update from remote peer 389a0c7a05d94b6abbba4db6e4b965a3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:12:55.573474 26097 raft_consensus.cc:1261] T 93602adfc5b841daac8dcd38836db5ed P 3efae221ecc84934b4f8ee12cd84f21b [term 1 FOLLOWER]: Refusing update from remote peer 389a0c7a05d94b6abbba4db6e4b965a3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:12:55.574537 26357 consensus_queue.cc:1035] T 93602adfc5b841daac8dcd38836db5ed P 389a0c7a05d94b6abbba4db6e4b965a3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1e026338e21940b1953ca8618702d3a7" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 38989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:12:55.574863 26364 consensus_queue.cc:1035] T 93602adfc5b841daac8dcd38836db5ed P 389a0c7a05d94b6abbba4db6e4b965a3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3efae221ecc84934b4f8ee12cd84f21b" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 33399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:12:55.600871 25738 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:56422:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:12:55.601208 25738 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:56422:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:12:55.604990 25738 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a2c1bf2caff94bc59213b9acede620e7: Sending DeleteTablet for 3 replicas of tablet 93602adfc5b841daac8dcd38836db5ed
I20260807 18:12:55.606577 25902 tablet_service.cc:1514] Processing DeleteTablet for tablet 93602adfc5b841daac8dcd38836db5ed with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:12:55 UTC) from {username='slave'} at 127.0.0.1:42402
I20260807 18:12:55.606597 26243 tablet_service.cc:1514] Processing DeleteTablet for tablet 93602adfc5b841daac8dcd38836db5ed with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:12:55 UTC) from {username='slave'} at 127.0.0.1:42654
I20260807 18:12:55.607427 26072 tablet_service.cc:1514] Processing DeleteTablet for tablet 93602adfc5b841daac8dcd38836db5ed with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:12:55 UTC) from {username='slave'} at 127.0.0.1:51720
I20260807 18:12:55.609263 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 25809
I20260807 18:12:55.617980 26383 tablet_replica.cc:330] T 93602adfc5b841daac8dcd38836db5ed P 389a0c7a05d94b6abbba4db6e4b965a3: stopping tablet replica
I20260807 18:12:55.618181 26384 tablet_replica.cc:330] T 93602adfc5b841daac8dcd38836db5ed P 3efae221ecc84934b4f8ee12cd84f21b: stopping tablet replica
I20260807 18:12:55.618651 26383 raft_consensus.cc:2229] T 93602adfc5b841daac8dcd38836db5ed P 389a0c7a05d94b6abbba4db6e4b965a3 [term 1 LEADER]: Raft consensus shutting down.
I20260807 18:12:55.618763 26384 raft_consensus.cc:2229] T 93602adfc5b841daac8dcd38836db5ed P 3efae221ecc84934b4f8ee12cd84f21b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:12:55.619441 26384 raft_consensus.cc:2258] T 93602adfc5b841daac8dcd38836db5ed P 3efae221ecc84934b4f8ee12cd84f21b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:12:55.619513 26383 raft_consensus.cc:2258] T 93602adfc5b841daac8dcd38836db5ed P 389a0c7a05d94b6abbba4db6e4b965a3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:12:55.622195 26384 ts_tablet_manager.cc:1905] T 93602adfc5b841daac8dcd38836db5ed P 3efae221ecc84934b4f8ee12cd84f21b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:12:55.623806 26345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:55.610226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42420 (local address 127.18.154.1:42857)
0807 18:12:55.610460 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0807 18:12:55.610470 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:12:55.620434 (+ 9964us) server_negotiation.cc:407] Connection header received
0807 18:12:55.620841 (+ 407us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:12:55.620850 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:12:55.620938 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:12:55.621033 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:12:55.622679 (+ 1646us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:12:55.622688 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:12:55.622696 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:12:55.622725 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:12:55.622776 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:12:55.622797 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:12:55.622945 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0807 18:12:55.623286 (+ 341us) server_negotiation.cc:299] Negotiation successful
0807 18:12:55.623371 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 18:12:55.625223 25982 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:12:55.625986 26382 tablet_replica.cc:330] T 93602adfc5b841daac8dcd38836db5ed P 1e026338e21940b1953ca8618702d3a7: stopping tablet replica
I20260807 18:12:55.626498 26382 raft_consensus.cc:2229] T 93602adfc5b841daac8dcd38836db5ed P 1e026338e21940b1953ca8618702d3a7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:12:55.627102 26382 raft_consensus.cc:2258] T 93602adfc5b841daac8dcd38836db5ed P 1e026338e21940b1953ca8618702d3a7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:12:55.629585 26382 ts_tablet_manager.cc:1905] T 93602adfc5b841daac8dcd38836db5ed P 1e026338e21940b1953ca8618702d3a7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:12:55.631224 26384 ts_tablet_manager.cc:1918] T 93602adfc5b841daac8dcd38836db5ed P 3efae221ecc84934b4f8ee12cd84f21b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:12:55.631418 26384 log.cc:1196] T 93602adfc5b841daac8dcd38836db5ed P 3efae221ecc84934b4f8ee12cd84f21b: Deleting WAL directory at /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126328808105-19048-0/minicluster-data/ts-1/wal/wals/93602adfc5b841daac8dcd38836db5ed
I20260807 18:12:55.632027 26384 ts_tablet_manager.cc:1939] T 93602adfc5b841daac8dcd38836db5ed P 3efae221ecc84934b4f8ee12cd84f21b: Deleting consensus metadata
I20260807 18:12:55.634054 25720 catalog_manager.cc:4841] TS 3efae221ecc84934b4f8ee12cd84f21b (127.18.154.2:33399): tablet 93602adfc5b841daac8dcd38836db5ed (table test-table [id=c06aa6a9eb4143faacb7847fd8b47337]) successfully deleted
I20260807 18:12:55.636044 26382 ts_tablet_manager.cc:1918] T 93602adfc5b841daac8dcd38836db5ed P 1e026338e21940b1953ca8618702d3a7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:12:55.636214 26382 log.cc:1196] T 93602adfc5b841daac8dcd38836db5ed P 1e026338e21940b1953ca8618702d3a7: Deleting WAL directory at /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786126328808105-19048-0/minicluster-data/ts-2/wal/wals/93602adfc5b841daac8dcd38836db5ed
I20260807 18:12:55.636699 26382 ts_tablet_manager.cc:1939] T 93602adfc5b841daac8dcd38836db5ed P 1e026338e21940b1953ca8618702d3a7: Deleting consensus metadata
I20260807 18:12:55.638655 25720 catalog_manager.cc:4841] TS 1e026338e21940b1953ca8618702d3a7 (127.18.154.3:38989): tablet 93602adfc5b841daac8dcd38836db5ed (table test-table [id=c06aa6a9eb4143faacb7847fd8b47337]) successfully deleted
I20260807 18:12:55.724740 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 25809
W20260807 18:12:55.749264 25719 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.154.1:42857 (error 108)
I20260807 18:12:55.749745 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 25994
I20260807 18:12:55.751776 26346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:55.751032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.1:42857 (local address 127.0.0.1:42432)
0807 18:12:55.751263 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0807 18:12:55.751496 (+ 233us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.1:42857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260807 18:12:55.752008 25719 catalog_manager.cc:4584] TS 389a0c7a05d94b6abbba4db6e4b965a3 (127.18.154.1:42857): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 93602adfc5b841daac8dcd38836db5ed: Network error: Client connection negotiation failed: client connection to 127.18.154.1:42857: connect: Connection refused (error 111)
I20260807 18:12:55.753340 26348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:55.750592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51732 (local address 127.18.154.2:33399)
0807 18:12:55.750785 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0807 18:12:55.750793 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:12:55.750911 (+ 118us) server_negotiation.cc:407] Connection header received
0807 18:12:55.751102 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:12:55.751110 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:12:55.751230 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:12:55.751345 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:12:55.752775 (+ 1430us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:12:55.752780 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:12:55.752784 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:12:55.752801 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:12:55.752833 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:12:55.752846 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:12:55.752956 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0807 18:12:55.753068 (+ 112us) server_negotiation.cc:299] Negotiation successful
0807 18:12:55.753118 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 18:12:55.754024 26148 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:12:55.805962 26346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:55.805329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.1:42857 (local address 127.0.0.1:42448)
0807 18:12:55.805668 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0807 18:12:55.805787 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.1:42857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140}
I20260807 18:12:55.807291 26346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:55.806827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.1:42857 (local address 127.0.0.1:42450)
0807 18:12:55.807036 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0807 18:12:55.807137 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.1:42857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260807 18:12:55.846045 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 25994
I20260807 18:12:55.868189 26346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:55.867674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.1:42857 (local address 127.0.0.1:42464)
0807 18:12:55.867920 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0807 18:12:55.868021 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.1:42857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
I20260807 18:12:55.869328 26346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:55.868954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.1:42857 (local address 127.0.0.1:42468)
0807 18:12:55.869116 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0807 18:12:55.869211 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.1:42857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260807 18:12:55.871567 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 26161
I20260807 18:12:55.874549 26343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:55.872344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42668 (local address 127.18.154.3:38989)
0807 18:12:55.872528 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0807 18:12:55.872533 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:12:55.872554 (+ 21us) server_negotiation.cc:407] Connection header received
0807 18:12:55.872635 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:12:55.872639 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:12:55.872721 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:12:55.872796 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:12:55.873890 (+ 1094us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:12:55.873894 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:12:55.873897 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:12:55.873912 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:12:55.873943 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:12:55.873955 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:12:55.874062 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0807 18:12:55.874150 (+ 88us) server_negotiation.cc:299] Negotiation successful
0807 18:12:55.874196 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260807 18:12:55.875255 26321 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:12:55.950256 26346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:55.949676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.1:42857 (local address 127.0.0.1:42480)
0807 18:12:55.949926 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0807 18:12:55.950051 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.1:42857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260807 18:12:55.951534 26346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:55.951040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.1:42857 (local address 127.0.0.1:42496)
0807 18:12:55.951273 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0807 18:12:55.951360 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.1:42857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120}
I20260807 18:12:55.965402 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 26161
I20260807 18:12:55.990556 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 25692
I20260807 18:12:55.991770 25777 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:12:56.104815 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 25692
2026-08-07T18:12:56Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (2847 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260807 18:12:56.155529 19048 mini_oidc.cc:152] Starting JWKS server
I20260807 18:12:56.158798 19048 webserver.cc:460] Webserver started at https://127.0.0.1:38061/ using document root <none> and password file <none>
I20260807 18:12:56.159179 19048 mini_oidc.cc:189] Starting OIDC Discovery server
I20260807 18:12:56.159904 19048 webserver.cc:460] Webserver started at http://127.0.0.1:42541/ using document root <none> and password file <none>
2026-08-07T18:12:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:12:56Z Disabled control of system clock
I20260807 18:12:56.196131 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786126328808105-19048-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786126328808105-19048-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786126328808105-19048-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.154.62:40183
--webserver_interface=127.18.154.62
--webserver_port=0
--builtin_ntp_servers=127.18.154.20:35739
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.154.62:40183
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38061/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786126328808105-19048-0/testchainca.cert with env {}
W20260807 18:12:56.542843 26400 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:12:56.543213 26400 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:12:56.543391 26400 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:12:56.550668 26400 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:12:56.550827 26400 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:12:56.550899 26400 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:12:56.550984 26400 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:12:56.551091 26400 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260807 18:12:56.551182 26400 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260807 18:12:56.551263 26400 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:38061/jwks.json
I20260807 18:12:56.561645 26400 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.154.20:35739
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786126328808105-19048-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786126328808105-19048-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.154.62:40183
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.154.62:40183
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38061/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.154.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786126328808105-19048-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786126328808105-19048-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:12:56.563143 26400 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:12:56.564464 26400 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:12:56.582213 26407 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:12:56.582216 26408 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:12:56.582105 26413 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:12:56.583313 26400 server_base.cc:1034] running on GCE node
I20260807 18:12:56.584333 26400 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:12:56.586324 26400 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:12:56.587620 26400 hybrid_clock.cc:648] HybridClock initialized: now 1786126376587519 us; error 83 us; skew 500 ppm
I20260807 18:12:56.588080 26400 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 18:12:56.596431 26400 webserver.cc:460] Webserver started at http://127.18.154.62:46743/ using document root <none> and password file <none>
I20260807 18:12:56.597239 26400 fs_manager.cc:359] Metadata directory not provided
I20260807 18:12:56.597386 26400 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:12:56.597812 26400 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:12:56.600955 26400 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786126328808105-19048-0/minicluster-data/master-0/data/instance:
uuid: "ccf7f97faaa146d69c2cf70e36add038"
format_stamp: "Formatted at 2026-08-07 18:12:56 on dist-test-slave-gkw7"
server_key: "aff2b73eeea304481448f58a78edd3f0"
server_key_iv: "02faa8aef2456224691fa69d0253893d"
server_key_version: "encryptionkey@0"
I20260807 18:12:56.601929 26400 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786126328808105-19048-0/minicluster-data/master-0/wal/instance:
uuid: "ccf7f97faaa146d69c2cf70e36add038"
format_stamp: "Formatted at 2026-08-07 18:12:56 on dist-test-slave-gkw7"
server_key: "aff2b73eeea304481448f58a78edd3f0"
server_key_iv: "02faa8aef2456224691fa69d0253893d"
server_key_version: "encryptionkey@0"
I20260807 18:12:56.607635 26400 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 18:12:56.611440 26422 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:12:56.613008 26400 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 18:12:56.613255 26400 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786126328808105-19048-0/minicluster-data/master-0/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786126328808105-19048-0/minicluster-data/master-0/wal
uuid: "ccf7f97faaa146d69c2cf70e36add038"
format_stamp: "Formatted at 2026-08-07 18:12:56 on dist-test-slave-gkw7"
server_key: "aff2b73eeea304481448f58a78edd3f0"
server_key_iv: "02faa8aef2456224691fa69d0253893d"
server_key_version: "encryptionkey@0"
I20260807 18:12:56.613484 26400 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786126328808105-19048-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786126328808105-19048-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786126328808105-19048-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:12:56.627184 26400 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:12:56.628038 26400 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:12:56.628367 26400 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:12:56.648512 26400 rpc_server.cc:307] RPC server started. Bound to: 127.18.154.62:40183
I20260807 18:12:56.648538 26492 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.154.62:40183 every 8 connection(s)
I20260807 18:12:56.650053 26400 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
I20260807 18:12:56.650561 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 26400
I20260807 18:12:56.650794 19048 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786126328808105-19048-0/minicluster-data/master-0/wal/instance
I20260807 18:12:56.651669 19048 external_mini_cluster.cc:1428] Setting key 85d89d14c4892e623e62dfa052c7f9da
I20260807 18:12:56.654882 26493 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:12:56.665820 26496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:56.653692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37114 (local address 127.18.154.62:40183)
0807 18:12:56.654707 (+ 1015us) server_negotiation.cc:206] Beginning negotiation
0807 18:12:56.654728 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0807 18:12:56.654771 (+ 43us) server_negotiation.cc:407] Connection header received
0807 18:12:56.656459 (+ 1688us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:12:56.656482 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:12:56.656955 (+ 473us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:12:56.657376 (+ 421us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:12:56.658018 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:56.659382 (+ 1364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:56.660857 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:56.661662 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:56.661894 (+ 232us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:56.663247 (+ 1353us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:12:56.663260 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:12:56.663280 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:12:56.663942 (+ 662us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:12:56.664034 (+ 92us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:12:56.664057 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:12:56.664204 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0807 18:12:56.664437 (+ 233us) server_negotiation.cc:299] Negotiation successful
0807 18:12:56.664772 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":561,"thread_start_us":349,"threads_started":1}
I20260807 18:12:56.666186 26493 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ccf7f97faaa146d69c2cf70e36add038: Bootstrap starting.
I20260807 18:12:56.669891 19048 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:12:56.669988 26493 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ccf7f97faaa146d69c2cf70e36add038: Neither blocks nor log segments found. Creating new log.
I20260807 18:12:56.671063 26493 log.cc:824] T 00000000000000000000000000000000 P ccf7f97faaa146d69c2cf70e36add038: Log is configured to *not* fsync() on all Append() calls
I20260807 18:12:56.674062 26493 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ccf7f97faaa146d69c2cf70e36add038: No bootstrap required, opened a new log
I20260807 18:12:56.681344 26493 raft_consensus.cc:348] T 00000000000000000000000000000000 P ccf7f97faaa146d69c2cf70e36add038 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ccf7f97faaa146d69c2cf70e36add038" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 40183 } }
I20260807 18:12:56.681820 26493 raft_consensus.cc:374] T 00000000000000000000000000000000 P ccf7f97faaa146d69c2cf70e36add038 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:12:56.681951 26493 raft_consensus.cc:729] T 00000000000000000000000000000000 P ccf7f97faaa146d69c2cf70e36add038 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccf7f97faaa146d69c2cf70e36add038, State: Initialized, Role: FOLLOWER
I20260807 18:12:56.682538 26493 consensus_queue.cc:260] T 00000000000000000000000000000000 P ccf7f97faaa146d69c2cf70e36add038 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ccf7f97faaa146d69c2cf70e36add038" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 40183 } }
I20260807 18:12:56.682786 26493 raft_consensus.cc:388] T 00000000000000000000000000000000 P ccf7f97faaa146d69c2cf70e36add038 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:12:56.682922 26493 raft_consensus.cc:482] T 00000000000000000000000000000000 P ccf7f97faaa146d69c2cf70e36add038 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:12:56.683115 26493 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ccf7f97faaa146d69c2cf70e36add038 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:12:56.685312 26493 raft_consensus.cc:504] T 00000000000000000000000000000000 P ccf7f97faaa146d69c2cf70e36add038 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ccf7f97faaa146d69c2cf70e36add038" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 40183 } }
I20260807 18:12:56.685882 26493 leader_election.cc:302] T 00000000000000000000000000000000 P ccf7f97faaa146d69c2cf70e36add038 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ccf7f97faaa146d69c2cf70e36add038; no voters:
I20260807 18:12:56.686465 26493 leader_election.cc:288] T 00000000000000000000000000000000 P ccf7f97faaa146d69c2cf70e36add038 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:12:56.686605 26500 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ccf7f97faaa146d69c2cf70e36add038 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:12:56.687355 26500 raft_consensus.cc:686] T 00000000000000000000000000000000 P ccf7f97faaa146d69c2cf70e36add038 [term 1 LEADER]: Becoming Leader. State: Replica: ccf7f97faaa146d69c2cf70e36add038, State: Running, Role: LEADER
I20260807 18:12:56.687855 26500 consensus_queue.cc:237] T 00000000000000000000000000000000 P ccf7f97faaa146d69c2cf70e36add038 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ccf7f97faaa146d69c2cf70e36add038" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 40183 } }
I20260807 18:12:56.688601 26493 sys_catalog.cc:564] T 00000000000000000000000000000000 P ccf7f97faaa146d69c2cf70e36add038 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:12:56.690896 26502 sys_catalog.cc:455] T 00000000000000000000000000000000 P ccf7f97faaa146d69c2cf70e36add038 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ccf7f97faaa146d69c2cf70e36add038. Latest consensus state: current_term: 1 leader_uuid: "ccf7f97faaa146d69c2cf70e36add038" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ccf7f97faaa146d69c2cf70e36add038" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 40183 } } }
I20260807 18:12:56.691424 26502 sys_catalog.cc:458] T 00000000000000000000000000000000 P ccf7f97faaa146d69c2cf70e36add038 [sys.catalog]: This master's current role is: LEADER
I20260807 18:12:56.692060 26501 sys_catalog.cc:455] T 00000000000000000000000000000000 P ccf7f97faaa146d69c2cf70e36add038 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ccf7f97faaa146d69c2cf70e36add038" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ccf7f97faaa146d69c2cf70e36add038" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 40183 } } }
I20260807 18:12:56.692410 26501 sys_catalog.cc:458] T 00000000000000000000000000000000 P ccf7f97faaa146d69c2cf70e36add038 [sys.catalog]: This master's current role is: LEADER
I20260807 18:12:56.694955 26514 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:12:56.699414 26514 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:12:56.707958 26514 catalog_manager.cc:1347] Generated new cluster ID: 25274436898c455cb8b83c2654371318
I20260807 18:12:56.708086 26514 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:12:56.732378 26514 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:12:56.733923 26514 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:12:56.741303 26514 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ccf7f97faaa146d69c2cf70e36add038: Generated new TSK 0
I20260807 18:12:56.742221 26514 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260807 18:12:56.789760 26538 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260807 18:12:56.790082 26538 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260807 18:12:56.791419 26496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:56.766887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37126 (local address 127.18.154.62:40183)
0807 18:12:56.767167 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0807 18:12:56.767173 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:12:56.767560 (+ 387us) server_negotiation.cc:407] Connection header received
0807 18:12:56.768013 (+ 453us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:12:56.768018 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:12:56.768148 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:12:56.768311 (+ 163us) server_negotiation.cc:226] Negotiated authn=JWT
0807 18:12:56.768994 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:56.770002 (+ 1008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:56.771689 (+ 1687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:56.772249 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:56.772423 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:56.773057 (+ 634us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0807 18:12:56.790595 (+ 17538us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38061/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0807 18:12:56.791026 (+ 431us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:37126: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38061/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":77}
W20260807 18:12:56.791561 26496 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:37126: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38061/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260807 18:12:56.793239 26536 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:12:56.766775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:40183 (local address 127.0.0.1:37126)
0807 18:12:56.767441 (+ 666us) negotiation.cc:107] Waiting for socket to connect
0807 18:12:56.767462 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 18:12:56.767676 (+ 214us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:12:56.768374 (+ 698us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:12:56.768379 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:12:56.768395 (+ 16us) client_negotiation.cc:189] Negotiated authn=JWT
0807 18:12:56.768804 (+ 409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:12:56.768817 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:12:56.770213 (+ 1396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:12:56.770227 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:12:56.771514 (+ 1287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:12:56.771527 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:12:56.772484 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:12:56.772492 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:12:56.772659 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:56.772734 (+ 75us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0807 18:12:56.790927 (+ 18193us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38061/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0807 18:12:56.791318 (+ 391us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.18.154.62:40183: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38061/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":312,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.18.154.62:40183: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38061/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260807 18:12:56.795397 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 26400
I20260807 18:12:56.796622 26486 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:12:56.897257 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 26400
2026-08-07T18:12:56Z chronyd exiting
I20260807 18:12:56.952999 19048 test_util.cc:175] -----------------------------------------------
I20260807 18:12:56.953137 19048 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786126328808105-19048-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (803 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260807 18:12:56.957278 19048 mini_oidc.cc:152] Starting JWKS server
I20260807 18:12:56.959270 19048 webserver.cc:460] Webserver started at https://127.0.0.1:39427/ using document root <none> and password file <none>
I20260807 18:12:56.959542 19048 mini_oidc.cc:189] Starting OIDC Discovery server
I20260807 18:12:56.960245 19048 webserver.cc:460] Webserver started at http://127.0.0.1:43789/ using document root <none> and password file <none>
2026-08-07T18:12:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:12:56Z Disabled control of system clock
I20260807 18:12:56.995298 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786126328808105-19048-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786126328808105-19048-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786126328808105-19048-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.154.62:38951
--webserver_interface=127.18.154.62
--webserver_port=0
--builtin_ntp_servers=127.18.154.20:37787
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.154.62:38951
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39427/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786126328808105-19048-0/testchainca.cert with env {}
W20260807 18:12:57.343118 26545 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:12:57.343556 26545 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:12:57.343758 26545 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:12:57.351639 26545 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:12:57.351835 26545 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:12:57.351945 26545 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:12:57.352082 26545 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:12:57.352173 26545 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260807 18:12:57.352277 26545 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260807 18:12:57.352365 26545 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:39427/jwks.json
I20260807 18:12:57.363390 26545 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.154.20:37787
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786126328808105-19048-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786126328808105-19048-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.154.62:38951
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.154.62:38951
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39427/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.154.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786126328808105-19048-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786126328808105-19048-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:12:57.365005 26545 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:12:57.366542 26545 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:12:57.380623 26553 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:12:57.380554 26555 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:12:57.381085 26552 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:12:57.381549 26545 server_base.cc:1034] running on GCE node
I20260807 18:12:57.382524 26545 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:12:57.384126 26545 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:12:57.385385 26545 hybrid_clock.cc:648] HybridClock initialized: now 1786126377385325 us; error 52 us; skew 500 ppm
I20260807 18:12:57.385818 26545 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 18:12:57.388906 26545 webserver.cc:460] Webserver started at http://127.18.154.62:46221/ using document root <none> and password file <none>
I20260807 18:12:57.389741 26545 fs_manager.cc:359] Metadata directory not provided
I20260807 18:12:57.389889 26545 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:12:57.390261 26545 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:12:57.394002 26545 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786126328808105-19048-0/minicluster-data/master-0/data/instance:
uuid: "4f1f68dc1b974965972af0c43107513d"
format_stamp: "Formatted at 2026-08-07 18:12:57 on dist-test-slave-gkw7"
server_key: "efac5e9ecf9c34a852e1bf886e2185d5"
server_key_iv: "58e8bb83b474cc6033f305916cea9bff"
server_key_version: "encryptionkey@0"
I20260807 18:12:57.395103 26545 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786126328808105-19048-0/minicluster-data/master-0/wal/instance:
uuid: "4f1f68dc1b974965972af0c43107513d"
format_stamp: "Formatted at 2026-08-07 18:12:57 on dist-test-slave-gkw7"
server_key: "efac5e9ecf9c34a852e1bf886e2185d5"
server_key_iv: "58e8bb83b474cc6033f305916cea9bff"
server_key_version: "encryptionkey@0"
I20260807 18:12:57.401268 26545 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260807 18:12:57.405334 26563 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:12:57.406899 26545 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:12:57.407119 26545 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786126328808105-19048-0/minicluster-data/master-0/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786126328808105-19048-0/minicluster-data/master-0/wal
uuid: "4f1f68dc1b974965972af0c43107513d"
format_stamp: "Formatted at 2026-08-07 18:12:57 on dist-test-slave-gkw7"
server_key: "efac5e9ecf9c34a852e1bf886e2185d5"
server_key_iv: "58e8bb83b474cc6033f305916cea9bff"
server_key_version: "encryptionkey@0"
I20260807 18:12:57.407425 26545 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786126328808105-19048-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786126328808105-19048-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786126328808105-19048-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:12:57.422127 26545 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:12:57.422995 26545 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:12:57.423341 26545 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:12:57.442749 26545 rpc_server.cc:307] RPC server started. Bound to: 127.18.154.62:38951
I20260807 18:12:57.442780 26632 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.154.62:38951 every 8 connection(s)
I20260807 18:12:57.444267 26545 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
I20260807 18:12:57.448169 26633 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:12:57.449973 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 26545
I20260807 18:12:57.450225 19048 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786126328808105-19048-0/minicluster-data/master-0/wal/instance
I20260807 18:12:57.451036 19048 external_mini_cluster.cc:1428] Setting key c58674b4e5b61e8278cb95a2440bafff
I20260807 18:12:57.462183 26633 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4f1f68dc1b974965972af0c43107513d: Bootstrap starting.
I20260807 18:12:57.465647 26637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:57.453252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49744 (local address 127.18.154.62:38951)
0807 18:12:57.454390 (+ 1138us) server_negotiation.cc:206] Beginning negotiation
0807 18:12:57.454401 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 18:12:57.454445 (+ 44us) server_negotiation.cc:407] Connection header received
0807 18:12:57.456045 (+ 1600us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:12:57.456069 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:12:57.456563 (+ 494us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:12:57.456970 (+ 407us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:12:57.457616 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:57.458908 (+ 1292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:57.460275 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:57.460992 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:57.461204 (+ 212us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:57.462529 (+ 1325us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:12:57.462544 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:12:57.462564 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:12:57.463322 (+ 758us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:12:57.463433 (+ 111us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:12:57.463464 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:12:57.463607 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0807 18:12:57.463850 (+ 243us) server_negotiation.cc:299] Negotiation successful
0807 18:12:57.464261 (+ 411us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":699,"thread_start_us":464,"threads_started":1}
I20260807 18:12:57.466604 26633 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4f1f68dc1b974965972af0c43107513d: Neither blocks nor log segments found. Creating new log.
I20260807 18:12:57.467797 26633 log.cc:824] T 00000000000000000000000000000000 P 4f1f68dc1b974965972af0c43107513d: Log is configured to *not* fsync() on all Append() calls
I20260807 18:12:57.469907 19048 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:12:57.471007 26633 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4f1f68dc1b974965972af0c43107513d: No bootstrap required, opened a new log
I20260807 18:12:57.478889 26633 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4f1f68dc1b974965972af0c43107513d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f1f68dc1b974965972af0c43107513d" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 38951 } }
I20260807 18:12:57.479387 26633 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4f1f68dc1b974965972af0c43107513d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:12:57.479522 26633 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4f1f68dc1b974965972af0c43107513d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f1f68dc1b974965972af0c43107513d, State: Initialized, Role: FOLLOWER
I20260807 18:12:57.480196 26633 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4f1f68dc1b974965972af0c43107513d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f1f68dc1b974965972af0c43107513d" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 38951 } }
I20260807 18:12:57.480445 26633 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4f1f68dc1b974965972af0c43107513d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:12:57.480602 26633 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4f1f68dc1b974965972af0c43107513d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:12:57.480772 26633 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4f1f68dc1b974965972af0c43107513d [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:12:57.483359 26633 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4f1f68dc1b974965972af0c43107513d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f1f68dc1b974965972af0c43107513d" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 38951 } }
I20260807 18:12:57.483902 26633 leader_election.cc:302] T 00000000000000000000000000000000 P 4f1f68dc1b974965972af0c43107513d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4f1f68dc1b974965972af0c43107513d; no voters:
I20260807 18:12:57.484555 26633 leader_election.cc:288] T 00000000000000000000000000000000 P 4f1f68dc1b974965972af0c43107513d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:12:57.484731 26640 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4f1f68dc1b974965972af0c43107513d [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:12:57.485826 26640 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4f1f68dc1b974965972af0c43107513d [term 1 LEADER]: Becoming Leader. State: Replica: 4f1f68dc1b974965972af0c43107513d, State: Running, Role: LEADER
I20260807 18:12:57.486389 26640 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4f1f68dc1b974965972af0c43107513d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f1f68dc1b974965972af0c43107513d" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 38951 } }
I20260807 18:12:57.487106 26633 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4f1f68dc1b974965972af0c43107513d [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:12:57.489904 26641 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4f1f68dc1b974965972af0c43107513d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4f1f68dc1b974965972af0c43107513d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f1f68dc1b974965972af0c43107513d" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 38951 } } }
I20260807 18:12:57.490324 26641 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4f1f68dc1b974965972af0c43107513d [sys.catalog]: This master's current role is: LEADER
I20260807 18:12:57.490798 26642 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4f1f68dc1b974965972af0c43107513d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4f1f68dc1b974965972af0c43107513d. Latest consensus state: current_term: 1 leader_uuid: "4f1f68dc1b974965972af0c43107513d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f1f68dc1b974965972af0c43107513d" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 38951 } } }
I20260807 18:12:57.491134 26642 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4f1f68dc1b974965972af0c43107513d [sys.catalog]: This master's current role is: LEADER
I20260807 18:12:57.491542 26652 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:12:57.497596 26652 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:12:57.506397 26652 catalog_manager.cc:1347] Generated new cluster ID: c1eda7cde2e4471683e3c5dbb1bb8150
I20260807 18:12:57.506536 26652 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:12:57.534965 26652 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:12:57.536623 26652 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:12:57.547771 26652 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4f1f68dc1b974965972af0c43107513d: Generated new TSK 0
I20260807 18:12:57.548988 26652 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260807 18:12:57.603103 26676 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260807 18:12:57.603253 26676 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260807 18:12:57.604319 26637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:57.576191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49754 (local address 127.18.154.62:38951)
0807 18:12:57.576432 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0807 18:12:57.576440 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:12:57.576840 (+ 400us) server_negotiation.cc:407] Connection header received
0807 18:12:57.577118 (+ 278us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:12:57.577122 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:12:57.577232 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:12:57.577382 (+ 150us) server_negotiation.cc:226] Negotiated authn=JWT
0807 18:12:57.578002 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:57.578803 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:57.580843 (+ 2040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:57.581337 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:57.581439 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:57.582101 (+ 662us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0807 18:12:57.603722 (+ 21621us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39427/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0807 18:12:57.604066 (+ 344us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:49754: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39427/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":87}
W20260807 18:12:57.604491 26637 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:49754: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39427/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260807 18:12:57.604753 26674 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:12:57.576015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:38951 (local address 127.0.0.1:49754)
0807 18:12:57.576692 (+ 677us) negotiation.cc:107] Waiting for socket to connect
0807 18:12:57.576737 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 18:12:57.576910 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:12:57.577426 (+ 516us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:12:57.577431 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:12:57.577446 (+ 15us) client_negotiation.cc:189] Negotiated authn=JWT
0807 18:12:57.577843 (+ 397us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:12:57.577856 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:12:57.578947 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:12:57.578953 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:12:57.580667 (+ 1714us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:12:57.580684 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:12:57.581520 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:12:57.581551 (+ 31us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:12:57.581770 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:57.581833 (+ 63us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0807 18:12:57.604006 (+ 22173us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39427/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0807 18:12:57.604326 (+ 320us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.18.154.62:38951: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39427/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":517,"thread_start_us":331,"threads_started":1}
I20260807 18:12:57.606079 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 26545
I20260807 18:12:57.607419 26627 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:12:57.704905 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 26545
2026-08-07T18:12:57Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (811 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260807 18:12:57.769110 19048 mini_oidc.cc:152] Starting JWKS server
I20260807 18:12:57.771179 19048 webserver.cc:460] Webserver started at https://127.0.0.1:42531/ using document root <none> and password file <none>
I20260807 18:12:57.771454 19048 mini_oidc.cc:189] Starting OIDC Discovery server
I20260807 18:12:57.772198 19048 webserver.cc:460] Webserver started at http://127.0.0.1:32823/ using document root <none> and password file <none>
2026-08-07T18:12:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:12:57Z Disabled control of system clock
I20260807 18:12:57.808876 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786126328808105-19048-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786126328808105-19048-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786126328808105-19048-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.154.62:44955
--webserver_interface=127.18.154.62
--webserver_port=0
--builtin_ntp_servers=127.18.154.20:46487
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.154.62:44955
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42531/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260807 18:12:58.166702 26687 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:12:58.167070 26687 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:12:58.167235 26687 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:12:58.174726 26687 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:12:58.174889 26687 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:12:58.174978 26687 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:12:58.175096 26687 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:12:58.175179 26687 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260807 18:12:58.175271 26687 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260807 18:12:58.175346 26687 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:42531/jwks.json
I20260807 18:12:58.185290 26687 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.154.20:46487
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786126328808105-19048-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786126328808105-19048-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.154.62:44955
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.154.62:44955
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42531/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.154.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786126328808105-19048-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:12:58.186780 26687 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:12:58.188714 26687 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:12:58.201252 26694 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:12:58.201269 26697 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:12:58.201177 26695 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:12:58.202170 26687 server_base.cc:1034] running on GCE node
I20260807 18:12:58.203117 26687 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:12:58.204691 26687 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:12:58.205968 26687 hybrid_clock.cc:648] HybridClock initialized: now 1786126378205902 us; error 53 us; skew 500 ppm
I20260807 18:12:58.206390 26687 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 18:12:58.208890 26687 webserver.cc:460] Webserver started at http://127.18.154.62:37341/ using document root <none> and password file <none>
I20260807 18:12:58.209733 26687 fs_manager.cc:359] Metadata directory not provided
I20260807 18:12:58.209875 26687 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:12:58.210223 26687 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:12:58.213497 26687 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786126328808105-19048-0/minicluster-data/master-0/data/instance:
uuid: "aef65552fda64fe6b757718548c8352b"
format_stamp: "Formatted at 2026-08-07 18:12:58 on dist-test-slave-gkw7"
server_key: "791ab0bbfa691a54813dc75f8e3699dd"
server_key_iv: "bf27502b56f362184743dd43684e2be9"
server_key_version: "encryptionkey@0"
I20260807 18:12:58.214526 26687 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786126328808105-19048-0/minicluster-data/master-0/wal/instance:
uuid: "aef65552fda64fe6b757718548c8352b"
format_stamp: "Formatted at 2026-08-07 18:12:58 on dist-test-slave-gkw7"
server_key: "791ab0bbfa691a54813dc75f8e3699dd"
server_key_iv: "bf27502b56f362184743dd43684e2be9"
server_key_version: "encryptionkey@0"
I20260807 18:12:58.220115 26687 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 18:12:58.223897 26705 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:12:58.225427 26687 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 18:12:58.225651 26687 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786126328808105-19048-0/minicluster-data/master-0/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786126328808105-19048-0/minicluster-data/master-0/wal
uuid: "aef65552fda64fe6b757718548c8352b"
format_stamp: "Formatted at 2026-08-07 18:12:58 on dist-test-slave-gkw7"
server_key: "791ab0bbfa691a54813dc75f8e3699dd"
server_key_iv: "bf27502b56f362184743dd43684e2be9"
server_key_version: "encryptionkey@0"
I20260807 18:12:58.225891 26687 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786126328808105-19048-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786126328808105-19048-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786126328808105-19048-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:12:58.252815 26687 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:12:58.253849 26687 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:12:58.254184 26687 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:12:58.273839 26687 rpc_server.cc:307] RPC server started. Bound to: 127.18.154.62:44955
I20260807 18:12:58.273859 26772 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.154.62:44955 every 8 connection(s)
I20260807 18:12:58.275277 26687 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
I20260807 18:12:58.279320 26773 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:12:58.285095 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 26687
I20260807 18:12:58.285329 19048 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786126328808105-19048-0/minicluster-data/master-0/wal/instance
I20260807 18:12:58.286058 19048 external_mini_cluster.cc:1428] Setting key 53309a91d043307eab17ed75a41cb3f7
I20260807 18:12:58.290831 26773 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aef65552fda64fe6b757718548c8352b: Bootstrap starting.
I20260807 18:12:58.294492 26773 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P aef65552fda64fe6b757718548c8352b: Neither blocks nor log segments found. Creating new log.
I20260807 18:12:58.295915 26773 log.cc:824] T 00000000000000000000000000000000 P aef65552fda64fe6b757718548c8352b: Log is configured to *not* fsync() on all Append() calls
I20260807 18:12:58.299404 26773 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aef65552fda64fe6b757718548c8352b: No bootstrap required, opened a new log
I20260807 18:12:58.301302 26779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:58.288269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43180 (local address 127.18.154.62:44955)
0807 18:12:58.289367 (+ 1098us) server_negotiation.cc:206] Beginning negotiation
0807 18:12:58.289390 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0807 18:12:58.289436 (+ 46us) server_negotiation.cc:407] Connection header received
0807 18:12:58.291291 (+ 1855us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:12:58.291317 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:12:58.291807 (+ 490us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:12:58.292153 (+ 346us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:12:58.292848 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:58.294237 (+ 1389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:58.296272 (+ 2035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:58.297092 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:58.297346 (+ 254us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:58.298799 (+ 1453us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:12:58.298806 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:12:58.298823 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:12:58.299388 (+ 565us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:12:58.299506 (+ 118us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:12:58.299544 (+ 38us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:12:58.299679 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0807 18:12:58.299926 (+ 247us) server_negotiation.cc:299] Negotiation successful
0807 18:12:58.300268 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":650,"thread_start_us":415,"threads_started":1}
I20260807 18:12:58.305430 19048 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:12:58.307885 26773 raft_consensus.cc:348] T 00000000000000000000000000000000 P aef65552fda64fe6b757718548c8352b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aef65552fda64fe6b757718548c8352b" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 44955 } }
I20260807 18:12:58.308266 26773 raft_consensus.cc:374] T 00000000000000000000000000000000 P aef65552fda64fe6b757718548c8352b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:12:58.308400 26773 raft_consensus.cc:729] T 00000000000000000000000000000000 P aef65552fda64fe6b757718548c8352b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aef65552fda64fe6b757718548c8352b, State: Initialized, Role: FOLLOWER
I20260807 18:12:58.309051 26773 consensus_queue.cc:260] T 00000000000000000000000000000000 P aef65552fda64fe6b757718548c8352b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aef65552fda64fe6b757718548c8352b" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 44955 } }
I20260807 18:12:58.309278 26773 raft_consensus.cc:388] T 00000000000000000000000000000000 P aef65552fda64fe6b757718548c8352b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:12:58.309413 26773 raft_consensus.cc:482] T 00000000000000000000000000000000 P aef65552fda64fe6b757718548c8352b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:12:58.309644 26773 raft_consensus.cc:3037] T 00000000000000000000000000000000 P aef65552fda64fe6b757718548c8352b [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:12:58.311875 26773 raft_consensus.cc:504] T 00000000000000000000000000000000 P aef65552fda64fe6b757718548c8352b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aef65552fda64fe6b757718548c8352b" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 44955 } }
I20260807 18:12:58.312407 26773 leader_election.cc:302] T 00000000000000000000000000000000 P aef65552fda64fe6b757718548c8352b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: aef65552fda64fe6b757718548c8352b; no voters:
I20260807 18:12:58.313202 26773 leader_election.cc:288] T 00000000000000000000000000000000 P aef65552fda64fe6b757718548c8352b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:12:58.313355 26781 raft_consensus.cc:2781] T 00000000000000000000000000000000 P aef65552fda64fe6b757718548c8352b [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:12:58.314172 26781 raft_consensus.cc:686] T 00000000000000000000000000000000 P aef65552fda64fe6b757718548c8352b [term 1 LEADER]: Becoming Leader. State: Replica: aef65552fda64fe6b757718548c8352b, State: Running, Role: LEADER
I20260807 18:12:58.314747 26781 consensus_queue.cc:237] T 00000000000000000000000000000000 P aef65552fda64fe6b757718548c8352b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aef65552fda64fe6b757718548c8352b" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 44955 } }
I20260807 18:12:58.315472 26773 sys_catalog.cc:564] T 00000000000000000000000000000000 P aef65552fda64fe6b757718548c8352b [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:12:58.317987 26782 sys_catalog.cc:455] T 00000000000000000000000000000000 P aef65552fda64fe6b757718548c8352b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "aef65552fda64fe6b757718548c8352b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aef65552fda64fe6b757718548c8352b" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 44955 } } }
I20260807 18:12:58.318452 26782 sys_catalog.cc:458] T 00000000000000000000000000000000 P aef65552fda64fe6b757718548c8352b [sys.catalog]: This master's current role is: LEADER
I20260807 18:12:58.318203 26783 sys_catalog.cc:455] T 00000000000000000000000000000000 P aef65552fda64fe6b757718548c8352b [sys.catalog]: SysCatalogTable state changed. Reason: New leader aef65552fda64fe6b757718548c8352b. Latest consensus state: current_term: 1 leader_uuid: "aef65552fda64fe6b757718548c8352b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aef65552fda64fe6b757718548c8352b" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 44955 } } }
I20260807 18:12:58.319970 26783 sys_catalog.cc:458] T 00000000000000000000000000000000 P aef65552fda64fe6b757718548c8352b [sys.catalog]: This master's current role is: LEADER
I20260807 18:12:58.320880 26789 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:12:58.326748 26789 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:12:58.336308 26789 catalog_manager.cc:1347] Generated new cluster ID: 0f02f1d9e30b42c0973a25568a2b69c2
I20260807 18:12:58.336443 26789 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:12:58.358237 26789 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:12:58.359592 26789 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:12:58.371518 26789 catalog_manager.cc:5868] T 00000000000000000000000000000000 P aef65552fda64fe6b757718548c8352b: Generated new TSK 0
I20260807 18:12:58.372515 26789 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:12:58.402693 26779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:58.392594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43194 (local address 127.18.154.62:44955)
0807 18:12:58.392849 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0807 18:12:58.392855 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:12:58.393183 (+ 328us) server_negotiation.cc:407] Connection header received
0807 18:12:58.393467 (+ 284us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:12:58.393471 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:12:58.393610 (+ 139us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:12:58.393788 (+ 178us) server_negotiation.cc:226] Negotiated authn=JWT
0807 18:12:58.394430 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:58.395744 (+ 1314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:58.397901 (+ 2157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:58.398470 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:58.398583 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:58.399301 (+ 718us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0807 18:12:58.401998 (+ 2697us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42531/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0807 18:12:58.402360 (+ 362us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:43194: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42531/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":115}
W20260807 18:12:58.402837 26779 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:43194: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42531/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260807 18:12:58.404821 26819 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:12:58.392395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:44955 (local address 127.0.0.1:43194)
0807 18:12:58.393062 (+ 667us) negotiation.cc:107] Waiting for socket to connect
0807 18:12:58.393083 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 18:12:58.393261 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:12:58.393829 (+ 568us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:12:58.393835 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:12:58.393850 (+ 15us) client_negotiation.cc:189] Negotiated authn=JWT
0807 18:12:58.394204 (+ 354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:12:58.394226 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:12:58.395911 (+ 1685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:12:58.395917 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:12:58.397740 (+ 1823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:12:58.397759 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:12:58.398686 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:12:58.398693 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:12:58.398890 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:58.398943 (+ 53us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0807 18:12:58.402273 (+ 3330us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42531/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0807 18:12:58.404352 (+ 2079us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.18.154.62:44955: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42531/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":510,"mutex_wait_us":1691,"thread_start_us":354,"threads_started":1}
I20260807 18:12:58.405982 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 26687
I20260807 18:12:58.407352 26766 generic_service.cc:196] Checking for leaks (request via RPC)
E20260807 18:12:58.415426 26821 webserver.cc:533] Webserver: sslize failed:
E20260807 18:12:58.415570 26821 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260807 18:12:58.503147 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 26687
2026-08-07T18:12:58Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (795 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-07T18:12:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:12:58Z Disabled control of system clock
I20260807 18:12:58.598677 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786126328808105-19048-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786126328808105-19048-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786126328808105-19048-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.154.62:36439
--webserver_interface=127.18.154.62
--webserver_port=0
--builtin_ntp_servers=127.18.154.20:39577
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.154.62:36439
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786126328808105-19048-0/testchainca.cert with env {}
W20260807 18:12:58.939980 26828 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:12:58.940304 26828 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:12:58.940395 26828 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:12:58.947613 26828 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:12:58.947732 26828 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:12:58.947769 26828 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:12:58.947827 26828 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:12:58.947871 26828 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260807 18:12:58.947940 26828 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260807 18:12:58.947983 26828 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260807 18:12:58.958201 26828 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.154.20:39577
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786126328808105-19048-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786126328808105-19048-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.154.62:36439
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.154.62:36439
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.154.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786126328808105-19048-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786126328808105-19048-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:12:58.959553 26828 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:12:58.960924 26828 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:12:58.973076 26838 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:12:58.973096 26839 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:12:58.973158 26842 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:12:58.974300 26828 server_base.cc:1034] running on GCE node
I20260807 18:12:58.975096 26828 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:12:58.976549 26828 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:12:58.977759 26828 hybrid_clock.cc:648] HybridClock initialized: now 1786126378977709 us; error 42 us; skew 500 ppm
I20260807 18:12:58.978127 26828 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 18:12:58.980544 26828 webserver.cc:460] Webserver started at http://127.18.154.62:38759/ using document root <none> and password file <none>
I20260807 18:12:58.981343 26828 fs_manager.cc:359] Metadata directory not provided
I20260807 18:12:58.981438 26828 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:12:58.981802 26828 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:12:58.984742 26828 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786126328808105-19048-0/minicluster-data/master-0/data/instance:
uuid: "176340d3ce5e4e3d9f17db11e9824448"
format_stamp: "Formatted at 2026-08-07 18:12:58 on dist-test-slave-gkw7"
server_key: "9dcb3bb3c3b6163fbaa3396d162fedfe"
server_key_iv: "6cbb17331c3af67df5aea714abe02d50"
server_key_version: "encryptionkey@0"
I20260807 18:12:58.985570 26828 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786126328808105-19048-0/minicluster-data/master-0/wal/instance:
uuid: "176340d3ce5e4e3d9f17db11e9824448"
format_stamp: "Formatted at 2026-08-07 18:12:58 on dist-test-slave-gkw7"
server_key: "9dcb3bb3c3b6163fbaa3396d162fedfe"
server_key_iv: "6cbb17331c3af67df5aea714abe02d50"
server_key_version: "encryptionkey@0"
I20260807 18:12:58.991237 26828 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260807 18:12:58.994982 26848 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:12:58.996529 26828 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 18:12:58.996769 26828 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786126328808105-19048-0/minicluster-data/master-0/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786126328808105-19048-0/minicluster-data/master-0/wal
uuid: "176340d3ce5e4e3d9f17db11e9824448"
format_stamp: "Formatted at 2026-08-07 18:12:58 on dist-test-slave-gkw7"
server_key: "9dcb3bb3c3b6163fbaa3396d162fedfe"
server_key_iv: "6cbb17331c3af67df5aea714abe02d50"
server_key_version: "encryptionkey@0"
I20260807 18:12:58.997014 26828 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786126328808105-19048-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786126328808105-19048-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786126328808105-19048-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:12:59.015722 26828 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:12:59.016659 26828 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:12:59.016966 26828 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:12:59.039902 26828 rpc_server.cc:307] RPC server started. Bound to: 127.18.154.62:36439
I20260807 18:12:59.039911 26916 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.154.62:36439 every 8 connection(s)
I20260807 18:12:59.041352 26828 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
I20260807 18:12:59.042300 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 26828
I20260807 18:12:59.042543 19048 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786126328808105-19048-0/minicluster-data/master-0/wal/instance
I20260807 18:12:59.043387 19048 external_mini_cluster.cc:1428] Setting key b7e11199e99c3c15908913473c05c7d4
I20260807 18:12:59.046945 26917 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:12:59.057462 26923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:59.045711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38076 (local address 127.18.154.62:36439)
0807 18:12:59.046784 (+ 1073us) server_negotiation.cc:206] Beginning negotiation
0807 18:12:59.046795 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 18:12:59.046839 (+ 44us) server_negotiation.cc:407] Connection header received
0807 18:12:59.048533 (+ 1694us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:12:59.048560 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:12:59.049050 (+ 490us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:12:59.049470 (+ 420us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:12:59.050128 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:59.051462 (+ 1334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:59.052804 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:59.053494 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:59.053740 (+ 246us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:59.055015 (+ 1275us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:12:59.055022 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:12:59.055037 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:12:59.055673 (+ 636us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:12:59.055739 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:12:59.055764 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:12:59.055898 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0807 18:12:59.056151 (+ 253us) server_negotiation.cc:299] Negotiation successful
0807 18:12:59.056488 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":585,"thread_start_us":352,"threads_started":1}
I20260807 18:12:59.058737 26917 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 176340d3ce5e4e3d9f17db11e9824448: Bootstrap starting.
I20260807 18:12:59.061872 19048 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:12:59.062695 26917 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 176340d3ce5e4e3d9f17db11e9824448: Neither blocks nor log segments found. Creating new log.
I20260807 18:12:59.063723 26917 log.cc:824] T 00000000000000000000000000000000 P 176340d3ce5e4e3d9f17db11e9824448: Log is configured to *not* fsync() on all Append() calls
I20260807 18:12:59.066793 26917 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 176340d3ce5e4e3d9f17db11e9824448: No bootstrap required, opened a new log
I20260807 18:12:59.074141 26917 raft_consensus.cc:348] T 00000000000000000000000000000000 P 176340d3ce5e4e3d9f17db11e9824448 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "176340d3ce5e4e3d9f17db11e9824448" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 36439 } }
I20260807 18:12:59.074546 26917 raft_consensus.cc:374] T 00000000000000000000000000000000 P 176340d3ce5e4e3d9f17db11e9824448 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:12:59.074666 26917 raft_consensus.cc:729] T 00000000000000000000000000000000 P 176340d3ce5e4e3d9f17db11e9824448 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 176340d3ce5e4e3d9f17db11e9824448, State: Initialized, Role: FOLLOWER
I20260807 18:12:59.075233 26917 consensus_queue.cc:260] T 00000000000000000000000000000000 P 176340d3ce5e4e3d9f17db11e9824448 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "176340d3ce5e4e3d9f17db11e9824448" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 36439 } }
I20260807 18:12:59.075482 26917 raft_consensus.cc:388] T 00000000000000000000000000000000 P 176340d3ce5e4e3d9f17db11e9824448 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:12:59.075625 26917 raft_consensus.cc:482] T 00000000000000000000000000000000 P 176340d3ce5e4e3d9f17db11e9824448 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:12:59.075814 26917 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 176340d3ce5e4e3d9f17db11e9824448 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:12:59.078794 26917 raft_consensus.cc:504] T 00000000000000000000000000000000 P 176340d3ce5e4e3d9f17db11e9824448 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "176340d3ce5e4e3d9f17db11e9824448" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 36439 } }
I20260807 18:12:59.079473 26917 leader_election.cc:302] T 00000000000000000000000000000000 P 176340d3ce5e4e3d9f17db11e9824448 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 176340d3ce5e4e3d9f17db11e9824448; no voters:
I20260807 18:12:59.080228 26917 leader_election.cc:288] T 00000000000000000000000000000000 P 176340d3ce5e4e3d9f17db11e9824448 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:12:59.080363 26925 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 176340d3ce5e4e3d9f17db11e9824448 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:12:59.081113 26925 raft_consensus.cc:686] T 00000000000000000000000000000000 P 176340d3ce5e4e3d9f17db11e9824448 [term 1 LEADER]: Becoming Leader. State: Replica: 176340d3ce5e4e3d9f17db11e9824448, State: Running, Role: LEADER
I20260807 18:12:59.081720 26925 consensus_queue.cc:237] T 00000000000000000000000000000000 P 176340d3ce5e4e3d9f17db11e9824448 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "176340d3ce5e4e3d9f17db11e9824448" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 36439 } }
I20260807 18:12:59.082497 26917 sys_catalog.cc:564] T 00000000000000000000000000000000 P 176340d3ce5e4e3d9f17db11e9824448 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:12:59.085920 26926 sys_catalog.cc:455] T 00000000000000000000000000000000 P 176340d3ce5e4e3d9f17db11e9824448 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "176340d3ce5e4e3d9f17db11e9824448" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "176340d3ce5e4e3d9f17db11e9824448" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 36439 } } }
I20260807 18:12:59.085975 26927 sys_catalog.cc:455] T 00000000000000000000000000000000 P 176340d3ce5e4e3d9f17db11e9824448 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 176340d3ce5e4e3d9f17db11e9824448. Latest consensus state: current_term: 1 leader_uuid: "176340d3ce5e4e3d9f17db11e9824448" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "176340d3ce5e4e3d9f17db11e9824448" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 36439 } } }
I20260807 18:12:59.086328 26926 sys_catalog.cc:458] T 00000000000000000000000000000000 P 176340d3ce5e4e3d9f17db11e9824448 [sys.catalog]: This master's current role is: LEADER
I20260807 18:12:59.086373 26927 sys_catalog.cc:458] T 00000000000000000000000000000000 P 176340d3ce5e4e3d9f17db11e9824448 [sys.catalog]: This master's current role is: LEADER
I20260807 18:12:59.094676 26940 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:12:59.099921 26940 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:12:59.108680 26940 catalog_manager.cc:1347] Generated new cluster ID: 1516c3e84342453a9643dd44fd531971
I20260807 18:12:59.108819 26940 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:12:59.141434 26940 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:12:59.142894 26940 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:12:59.161108 26940 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 176340d3ce5e4e3d9f17db11e9824448: Generated new TSK 0
I20260807 18:12:59.162138 26940 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:12:59.310323 26923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:12:59.183926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38080 (local address 127.18.154.62:36439)
0807 18:12:59.184290 (+ 364us) server_negotiation.cc:206] Beginning negotiation
0807 18:12:59.184302 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 18:12:59.185070 (+ 768us) server_negotiation.cc:407] Connection header received
0807 18:12:59.185377 (+ 307us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:12:59.185381 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:12:59.185489 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:12:59.185669 (+ 180us) server_negotiation.cc:226] Negotiated authn=JWT
0807 18:12:59.186285 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:59.187072 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:59.189143 (+ 2071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:12:59.189696 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:12:59.189814 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:59.190411 (+ 597us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0807 18:12:59.309724 (+119313us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0807 18:12:59.310086 (+ 362us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:38080: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":167}
W20260807 18:12:59.310493 26923 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:38080: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260807 18:12:59.310875 26956 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:12:59.183780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:36439 (local address 127.0.0.1:38080)
0807 18:12:59.184934 (+ 1154us) negotiation.cc:107] Waiting for socket to connect
0807 18:12:59.184955 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 18:12:59.185153 (+ 198us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:12:59.185713 (+ 560us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:12:59.185719 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:12:59.185733 (+ 14us) client_negotiation.cc:189] Negotiated authn=JWT
0807 18:12:59.186060 (+ 327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:12:59.186072 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:12:59.187229 (+ 1157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:12:59.187235 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:12:59.188980 (+ 1745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:12:59.189001 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:12:59.189900 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:12:59.189907 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:12:59.190106 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:12:59.190155 (+ 49us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0807 18:12:59.310050 (+119895us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0807 18:12:59.310430 (+ 380us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.18.154.62:36439: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":950,"thread_start_us":721,"threads_started":1}
I20260807 18:12:59.311920 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 26828
I20260807 18:12:59.313169 26912 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:12:59.416451 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 26828
2026-08-07T18:12:59Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (895 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (435 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (455 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126328808105-19048-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:13:00 dist-test-slave-gkw7 krb5kdc[26979](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:13:00 dist-test-slave-gkw7 krb5kdc[26979](info): set up 2 sockets
Aug 07 18:13:00 dist-test-slave-gkw7 krb5kdc[26979](info): commencing operation
krb5kdc: starting...
W20260807 18:13:01.590150 19048 mini_kdc.cc:121] Time spent starting KDC: real 1.238s user 0.009s sys 0.019s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126328808105-19048-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126328808105-19048-0/krb5kdc/test-user.keytab.
Aug 07 18:13:01 dist-test-slave-gkw7 krb5kdc[26979](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126381, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:13:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:13:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.154.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.154.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.154.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.62.keytab.
Entry for principal kudu/127.18.154.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.62.keytab.
I20260807 18:13:01.855446 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126328808105-19048-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126328808105-19048-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126328808105-19048-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.154.62:40319
--webserver_interface=127.18.154.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.62.keytab
--principal=kudu/127.18.154.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.154.20:35225
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.154.62:40319
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126328808105-19048-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126328808105-19048-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126328808105-19048-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:13:02.209146 26994 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:13:02.209486 26994 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:13:02.209688 26994 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:13:02.217088 26994 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:13:02.217226 26994 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:13:02.217331 26994 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:13:02.217419 26994 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:13:02.217492 26994 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:13:02.227697 26994 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.154.20:35225
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126328808105-19048-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126328808105-19048-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.154.62:40319
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.62.keytab
--principal=kudu/127.18.154.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.154.62:40319
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.154.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126328808105-19048-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:13:02.229101 26994 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:13:02.230540 26994 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:13:02.243453 27006 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:13:02.243526 27008 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:13:02.243526 27005 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:13:02.244457 26994 server_base.cc:1034] running on GCE node
I20260807 18:13:02.245304 26994 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:13:02.246842 26994 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:13:02.248138 26994 hybrid_clock.cc:648] HybridClock initialized: now 1786126382248078 us; error 34 us; skew 500 ppm
I20260807 18:13:02.248567 26994 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:13:02 dist-test-slave-gkw7 krb5kdc[26979](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126382, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:13:02.256795 26994 init.cc:377] Logged in from keytab as kudu/127.18.154.62@KRBTEST.COM (short username kudu)
I20260807 18:13:02.258925 26994 webserver.cc:460] Webserver started at http://127.18.154.62:38133/ using document root <none> and password file <none>
I20260807 18:13:02.259936 26994 fs_manager.cc:359] Metadata directory not provided
I20260807 18:13:02.260115 26994 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:13:02.260468 26994 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:13:02.263978 26994 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126328808105-19048-0/minicluster-data/master-0/data/instance:
uuid: "a8ed996dc840467ba43ab6739aae2575"
format_stamp: "Formatted at 2026-08-07 18:13:02 on dist-test-slave-gkw7"
server_key: "cb36962a41ba248cc06e0ae37f83519b"
server_key_iv: "dc81f9445833c616554b5a643940cc66"
server_key_version: "encryptionkey@0"
I20260807 18:13:02.264837 26994 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126328808105-19048-0/minicluster-data/master-0/wal/instance:
uuid: "a8ed996dc840467ba43ab6739aae2575"
format_stamp: "Formatted at 2026-08-07 18:13:02 on dist-test-slave-gkw7"
server_key: "cb36962a41ba248cc06e0ae37f83519b"
server_key_iv: "dc81f9445833c616554b5a643940cc66"
server_key_version: "encryptionkey@0"
I20260807 18:13:02.270876 26994 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.004s
I20260807 18:13:02.274904 27016 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:13:02.276484 26994 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 18:13:02.276687 26994 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126328808105-19048-0/minicluster-data/master-0/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126328808105-19048-0/minicluster-data/master-0/wal
uuid: "a8ed996dc840467ba43ab6739aae2575"
format_stamp: "Formatted at 2026-08-07 18:13:02 on dist-test-slave-gkw7"
server_key: "cb36962a41ba248cc06e0ae37f83519b"
server_key_iv: "dc81f9445833c616554b5a643940cc66"
server_key_version: "encryptionkey@0"
I20260807 18:13:02.276911 26994 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126328808105-19048-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126328808105-19048-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126328808105-19048-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:13:02.297709 26994 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:13:02.302865 26994 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:13:02.303200 26994 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:13:02.322405 26994 rpc_server.cc:307] RPC server started. Bound to: 127.18.154.62:40319
I20260807 18:13:02.322463 27086 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.154.62:40319 every 8 connection(s)
I20260807 18:13:02.323912 26994 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
I20260807 18:13:02.327881 27087 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:13:02.332362 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 26994
I20260807 18:13:02.332620 19048 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126328808105-19048-0/minicluster-data/master-0/wal/instance
I20260807 18:13:02.333461 19048 external_mini_cluster.cc:1428] Setting key e11cbc006b900ea6ea4420c955a97bb1
I20260807 18:13:02.342518 27087 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a8ed996dc840467ba43ab6739aae2575: Bootstrap starting.
I20260807 18:13:02.346593 27087 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a8ed996dc840467ba43ab6739aae2575: Neither blocks nor log segments found. Creating new log.
I20260807 18:13:02.348059 27087 log.cc:824] T 00000000000000000000000000000000 P a8ed996dc840467ba43ab6739aae2575: Log is configured to *not* fsync() on all Append() calls
Aug 07 18:13:02 dist-test-slave-gkw7 krb5kdc[26979](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126381, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.154.62@KRBTEST.COM
I20260807 18:13:02.351145 27087 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a8ed996dc840467ba43ab6739aae2575: No bootstrap required, opened a new log
I20260807 18:13:02.358685 27091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:02.335676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51714 (local address 127.18.154.62:40319)
0807 18:13:02.336451 (+ 775us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:02.336463 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:02.336503 (+ 40us) server_negotiation.cc:407] Connection header received
0807 18:13:02.338212 (+ 1709us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:02.338235 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:02.338670 (+ 435us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:02.338994 (+ 324us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:02.340864 (+ 1870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:02.342327 (+ 1463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:02.344089 (+ 1762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:02.344887 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:02.345057 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:02.350000 (+ 4943us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:13:02.350009 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:13:02.350065 (+ 56us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:13:02.350108 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:13:02.353886 (+ 3778us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:02.354739 (+ 853us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:02.354747 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:02.354755 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:02.354968 (+ 213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:02.355409 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:02.355415 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:02.355418 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:02.356099 (+ 681us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:13:02.356454 (+ 355us) server_negotiation.cc:1035] Waiting for connection context
0807 18:13:02.357003 (+ 549us) server_negotiation.cc:299] Negotiation successful
0807 18:13:02.357496 (+ 493us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":556,"thread_start_us":336,"threads_started":1}
I20260807 18:13:02.359191 27087 raft_consensus.cc:348] T 00000000000000000000000000000000 P a8ed996dc840467ba43ab6739aae2575 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8ed996dc840467ba43ab6739aae2575" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 40319 } }
I20260807 18:13:02.359613 27087 raft_consensus.cc:374] T 00000000000000000000000000000000 P a8ed996dc840467ba43ab6739aae2575 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:13:02.359730 27087 raft_consensus.cc:729] T 00000000000000000000000000000000 P a8ed996dc840467ba43ab6739aae2575 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8ed996dc840467ba43ab6739aae2575, State: Initialized, Role: FOLLOWER
I20260807 18:13:02.360313 27087 consensus_queue.cc:260] T 00000000000000000000000000000000 P a8ed996dc840467ba43ab6739aae2575 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8ed996dc840467ba43ab6739aae2575" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 40319 } }
I20260807 18:13:02.360581 27087 raft_consensus.cc:388] T 00000000000000000000000000000000 P a8ed996dc840467ba43ab6739aae2575 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:13:02.360730 27087 raft_consensus.cc:482] T 00000000000000000000000000000000 P a8ed996dc840467ba43ab6739aae2575 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:13:02.360920 27087 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a8ed996dc840467ba43ab6739aae2575 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:13:02.363504 27087 raft_consensus.cc:504] T 00000000000000000000000000000000 P a8ed996dc840467ba43ab6739aae2575 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8ed996dc840467ba43ab6739aae2575" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 40319 } }
I20260807 18:13:02.364015 27087 leader_election.cc:302] T 00000000000000000000000000000000 P a8ed996dc840467ba43ab6739aae2575 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a8ed996dc840467ba43ab6739aae2575; no voters:
I20260807 18:13:02.364638 27087 leader_election.cc:288] T 00000000000000000000000000000000 P a8ed996dc840467ba43ab6739aae2575 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:13:02.364809 27096 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a8ed996dc840467ba43ab6739aae2575 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:13:02.365513 27096 raft_consensus.cc:686] T 00000000000000000000000000000000 P a8ed996dc840467ba43ab6739aae2575 [term 1 LEADER]: Becoming Leader. State: Replica: a8ed996dc840467ba43ab6739aae2575, State: Running, Role: LEADER
I20260807 18:13:02.366096 27096 consensus_queue.cc:237] T 00000000000000000000000000000000 P a8ed996dc840467ba43ab6739aae2575 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8ed996dc840467ba43ab6739aae2575" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 40319 } }
I20260807 18:13:02.366925 27087 sys_catalog.cc:564] T 00000000000000000000000000000000 P a8ed996dc840467ba43ab6739aae2575 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:13:02.369033 27098 sys_catalog.cc:455] T 00000000000000000000000000000000 P a8ed996dc840467ba43ab6739aae2575 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a8ed996dc840467ba43ab6739aae2575" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8ed996dc840467ba43ab6739aae2575" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 40319 } } }
I20260807 18:13:02.369002 27099 sys_catalog.cc:455] T 00000000000000000000000000000000 P a8ed996dc840467ba43ab6739aae2575 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a8ed996dc840467ba43ab6739aae2575. Latest consensus state: current_term: 1 leader_uuid: "a8ed996dc840467ba43ab6739aae2575" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8ed996dc840467ba43ab6739aae2575" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 40319 } } }
I20260807 18:13:02.369462 27098 sys_catalog.cc:458] T 00000000000000000000000000000000 P a8ed996dc840467ba43ab6739aae2575 [sys.catalog]: This master's current role is: LEADER
I20260807 18:13:02.369613 27099 sys_catalog.cc:458] T 00000000000000000000000000000000 P a8ed996dc840467ba43ab6739aae2575 [sys.catalog]: This master's current role is: LEADER
I20260807 18:13:02.371084 27102 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:13:02.375656 27102 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:13:02.384869 27102 catalog_manager.cc:1347] Generated new cluster ID: a969f5a449474afa91551016309dadb5
I20260807 18:13:02.384985 27102 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:13:02.396836 27102 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:13:02.398355 27102 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:13:02.415655 27102 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a8ed996dc840467ba43ab6739aae2575: Generated new TSK 0
I20260807 18:13:02.416680 27102 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.154.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.154.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.154.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.1.keytab.
Entry for principal kudu/127.18.154.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.1.keytab.
I20260807 18:13:02.481515 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126328808105-19048-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126328808105-19048-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126328808105-19048-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126328808105-19048-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.154.1:0
--local_ip_for_outbound_sockets=127.18.154.1
--webserver_interface=127.18.154.1
--webserver_port=0
--tserver_master_addrs=127.18.154.62:40319
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.1.keytab
--principal=kudu/127.18.154.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.154.20:35225
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126328808105-19048-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126328808105-19048-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126328808105-19048-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:13:02.828679 27121 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:13:02.829036 27121 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:13:02.829209 27121 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:13:02.836566 27121 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:13:02.836725 27121 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:13:02.836901 27121 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.154.1
I20260807 18:13:02.847007 27121 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.154.20:35225
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126328808105-19048-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126328808105-19048-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.1.keytab
--principal=kudu/127.18.154.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.154.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126328808105-19048-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.154.1
--webserver_port=0
--tserver_master_addrs=127.18.154.62:40319
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.154.1
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126328808105-19048-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:13:02.848372 27121 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:13:02.849763 27121 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:13:02.863708 27128 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:13:02.863725 27131 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:13:02.863934 27129 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:13:02.864538 27121 server_base.cc:1034] running on GCE node
I20260807 18:13:02.865231 27121 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:13:02.866338 27121 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:13:02.867589 27121 hybrid_clock.cc:648] HybridClock initialized: now 1786126382867531 us; error 53 us; skew 500 ppm
I20260807 18:13:02.867959 27121 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:13:02 dist-test-slave-gkw7 krb5kdc[26979](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126382, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:13:02.871556 27121 init.cc:377] Logged in from keytab as kudu/127.18.154.1@KRBTEST.COM (short username kudu)
I20260807 18:13:02.873152 27121 webserver.cc:460] Webserver started at http://127.18.154.1:32861/ using document root <none> and password file <none>
I20260807 18:13:02.873968 27121 fs_manager.cc:359] Metadata directory not provided
I20260807 18:13:02.874102 27121 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:13:02.874449 27121 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:13:02.877537 27121 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126328808105-19048-0/minicluster-data/ts-0/data/instance:
uuid: "41592426f01f4920833ae9ba37810ab8"
format_stamp: "Formatted at 2026-08-07 18:13:02 on dist-test-slave-gkw7"
server_key: "f08671e243cc260cb888f17f4c74d0cf"
server_key_iv: "171b63cb0c8fcd8a2160acce8dfa072d"
server_key_version: "encryptionkey@0"
I20260807 18:13:02.878448 27121 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126328808105-19048-0/minicluster-data/ts-0/wal/instance:
uuid: "41592426f01f4920833ae9ba37810ab8"
format_stamp: "Formatted at 2026-08-07 18:13:02 on dist-test-slave-gkw7"
server_key: "f08671e243cc260cb888f17f4c74d0cf"
server_key_iv: "171b63cb0c8fcd8a2160acce8dfa072d"
server_key_version: "encryptionkey@0"
I20260807 18:13:02.884546 27121 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260807 18:13:02.888417 27143 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:13:02.890116 27121 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:13:02.890303 27121 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126328808105-19048-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126328808105-19048-0/minicluster-data/ts-0/wal
uuid: "41592426f01f4920833ae9ba37810ab8"
format_stamp: "Formatted at 2026-08-07 18:13:02 on dist-test-slave-gkw7"
server_key: "f08671e243cc260cb888f17f4c74d0cf"
server_key_iv: "171b63cb0c8fcd8a2160acce8dfa072d"
server_key_version: "encryptionkey@0"
I20260807 18:13:02.890524 27121 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126328808105-19048-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126328808105-19048-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126328808105-19048-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:13:02.910987 27121 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:13:02.916026 27121 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:13:02.916396 27121 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:13:02.917469 27121 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:13:02.918908 27121 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:13:02.919018 27121 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:13:02.919170 27121 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:13:02.919240 27121 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:13:02.952361 27121 rpc_server.cc:307] RPC server started. Bound to: 127.18.154.1:43033
I20260807 18:13:02.952396 27298 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.154.1:43033 every 8 connection(s)
I20260807 18:13:02.953845 27121 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126328808105-19048-0/minicluster-data/ts-0/data/info.pb
I20260807 18:13:02.959854 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 27121
I20260807 18:13:02.960081 19048 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126328808105-19048-0/minicluster-data/ts-0/wal/instance
I20260807 18:13:02.960732 19048 external_mini_cluster.cc:1428] Setting key daac5bc869e60c2692a2db55665efae5
Aug 07 18:13:02 dist-test-slave-gkw7 krb5kdc[26979](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126382, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.1@KRBTEST.COM for kudu/127.18.154.62@KRBTEST.COM
I20260807 18:13:02.976778 27301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:02.956710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.1:47905 (local address 127.18.154.62:40319)
0807 18:13:02.957212 (+ 502us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:02.957217 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:02.958232 (+ 1015us) server_negotiation.cc:407] Connection header received
0807 18:13:02.960186 (+ 1954us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:02.960192 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:02.960302 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:02.960428 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:02.962343 (+ 1915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:02.963385 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:02.964952 (+ 1567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:02.965812 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:02.965967 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:02.970006 (+ 4039us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:13:02.970015 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:13:02.970024 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:13:02.970074 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:13:02.973109 (+ 3035us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:02.974017 (+ 908us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:02.974035 (+ 18us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:02.974041 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:02.974202 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:02.974674 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:02.974682 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:02.974687 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:02.975068 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:13:02.975233 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0807 18:13:02.976001 (+ 768us) server_negotiation.cc:299] Negotiation successful
0807 18:13:02.976302 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":391,"thread_start_us":282,"threads_started":1}
I20260807 18:13:02.977636 27303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:02.957206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:40319 (local address 127.18.154.1:47905)
0807 18:13:02.958051 (+ 845us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:02.958102 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0807 18:13:02.959857 (+ 1755us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:13:02.960689 (+ 832us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:13:02.960697 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:13:02.961301 (+ 604us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:13:02.962164 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:02.962190 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:02.963542 (+ 1352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:02.963546 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:02.964798 (+ 1252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:02.964810 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:02.966107 (+ 1297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:02.966113 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:02.966357 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:02.967181 (+ 824us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:13:02.967226 (+ 45us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:13:02.969746 (+ 2520us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:13:02.973261 (+ 3515us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:13:02.973274 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:13:02.973297 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:13:02.973875 (+ 578us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:13:02.974342 (+ 467us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:13:02.974345 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:13:02.974348 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:13:02.974548 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:13:02.975200 (+ 652us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:13:02.975206 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:13:02.975573 (+ 367us) client_negotiation.cc:769] Sending connection context
0807 18:13:02.975837 (+ 264us) client_negotiation.cc:240] Negotiation successful
0807 18:13:02.976334 (+ 497us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":593,"thread_start_us":386,"threads_started":1}
I20260807 18:13:02.979385 27299 heartbeater.cc:347] Connected to a master server at 127.18.154.62:40319
I20260807 18:13:02.979691 27299 heartbeater.cc:464] Registering TS with master...
I20260807 18:13:02.980531 27299 heartbeater.cc:511] Master 127.18.154.62:40319 requested a full tablet report, sending...
I20260807 18:13:02.982889 27041 ts_manager.cc:194] Registered new tserver with Master: 41592426f01f4920833ae9ba37810ab8 (127.18.154.1:43033)
I20260807 18:13:02.984476 27041 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.154.1@KRBTEST.COM'} at 127.18.154.1:47905
WARNING: no policy specified for kudu/127.18.154.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.154.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.154.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.2.keytab.
Entry for principal kudu/127.18.154.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.2.keytab.
I20260807 18:13:03.021415 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126328808105-19048-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126328808105-19048-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126328808105-19048-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126328808105-19048-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.154.2:0
--local_ip_for_outbound_sockets=127.18.154.2
--webserver_interface=127.18.154.2
--webserver_port=0
--tserver_master_addrs=127.18.154.62:40319
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.2.keytab
--principal=kudu/127.18.154.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.154.20:35225
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126328808105-19048-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126328808105-19048-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126328808105-19048-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:13:03.381853 27312 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:13:03.382220 27312 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:13:03.382365 27312 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:13:03.390107 27312 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:13:03.390233 27312 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:13:03.390412 27312 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.154.2
I20260807 18:13:03.400477 27312 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.154.20:35225
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126328808105-19048-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126328808105-19048-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.2.keytab
--principal=kudu/127.18.154.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.154.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126328808105-19048-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.154.2
--webserver_port=0
--tserver_master_addrs=127.18.154.62:40319
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.154.2
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126328808105-19048-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:13:03.401908 27312 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:13:03.403241 27312 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:13:03.416811 27320 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:13:03.416918 27321 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:13:03.416955 27323 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:13:03.418210 27312 server_base.cc:1034] running on GCE node
I20260807 18:13:03.418787 27312 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:13:03.419680 27312 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:13:03.420881 27312 hybrid_clock.cc:648] HybridClock initialized: now 1786126383420837 us; error 31 us; skew 500 ppm
I20260807 18:13:03.421175 27312 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:13:03 dist-test-slave-gkw7 krb5kdc[26979](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126383, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:13:03.424377 27312 init.cc:377] Logged in from keytab as kudu/127.18.154.2@KRBTEST.COM (short username kudu)
I20260807 18:13:03.426025 27312 webserver.cc:460] Webserver started at http://127.18.154.2:45081/ using document root <none> and password file <none>
I20260807 18:13:03.426759 27312 fs_manager.cc:359] Metadata directory not provided
I20260807 18:13:03.426908 27312 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:13:03.427235 27312 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:13:03.430963 27312 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126328808105-19048-0/minicluster-data/ts-1/data/instance:
uuid: "5e1f7fd1245c43b9b1706676d2c1f0db"
format_stamp: "Formatted at 2026-08-07 18:13:03 on dist-test-slave-gkw7"
server_key: "00163dbabb80dcf00503f160bb7cdad8"
server_key_iv: "acd9e214648f8ba012a7f8a47edb3cf8"
server_key_version: "encryptionkey@0"
I20260807 18:13:03.431870 27312 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126328808105-19048-0/minicluster-data/ts-1/wal/instance:
uuid: "5e1f7fd1245c43b9b1706676d2c1f0db"
format_stamp: "Formatted at 2026-08-07 18:13:03 on dist-test-slave-gkw7"
server_key: "00163dbabb80dcf00503f160bb7cdad8"
server_key_iv: "acd9e214648f8ba012a7f8a47edb3cf8"
server_key_version: "encryptionkey@0"
I20260807 18:13:03.437450 27312 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260807 18:13:03.441160 27333 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:13:03.442590 27312 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:13:03.442749 27312 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126328808105-19048-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126328808105-19048-0/minicluster-data/ts-1/wal
uuid: "5e1f7fd1245c43b9b1706676d2c1f0db"
format_stamp: "Formatted at 2026-08-07 18:13:03 on dist-test-slave-gkw7"
server_key: "00163dbabb80dcf00503f160bb7cdad8"
server_key_iv: "acd9e214648f8ba012a7f8a47edb3cf8"
server_key_version: "encryptionkey@0"
I20260807 18:13:03.442936 27312 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126328808105-19048-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126328808105-19048-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126328808105-19048-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:13:03.459996 27312 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:13:03.464768 27312 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:13:03.465034 27312 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:13:03.466120 27312 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:13:03.467650 27312 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:13:03.467767 27312 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:13:03.467892 27312 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:13:03.467957 27312 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:13:03.506326 27312 rpc_server.cc:307] RPC server started. Bound to: 127.18.154.2:45645
I20260807 18:13:03.506345 27483 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.154.2:45645 every 8 connection(s)
I20260807 18:13:03.507787 27312 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126328808105-19048-0/minicluster-data/ts-1/data/info.pb
I20260807 18:13:03.508530 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 27312
I20260807 18:13:03.508795 19048 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126328808105-19048-0/minicluster-data/ts-1/wal/instance
I20260807 18:13:03.509497 19048 external_mini_cluster.cc:1428] Setting key 2a3c179091aaf6da2f29db4a9156f0f2
Aug 07 18:13:03 dist-test-slave-gkw7 krb5kdc[26979](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126383, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.2@KRBTEST.COM for kudu/127.18.154.62@KRBTEST.COM
I20260807 18:13:03.531579 27486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:03.510829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.2:37867 (local address 127.18.154.62:40319)
0807 18:13:03.511441 (+ 612us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:03.511445 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:03.512805 (+ 1360us) server_negotiation.cc:407] Connection header received
0807 18:13:03.514960 (+ 2155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:03.514966 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:03.515063 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:03.515199 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:03.516917 (+ 1718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:03.517839 (+ 922us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:03.519595 (+ 1756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:03.520112 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:03.520265 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:03.524945 (+ 4680us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:13:03.524951 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:13:03.524954 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:13:03.524994 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:13:03.528086 (+ 3092us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:03.528971 (+ 885us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:03.528980 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:03.528985 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:03.529138 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:03.529676 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:03.529685 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:03.529688 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:03.529979 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:13:03.530146 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0807 18:13:03.530923 (+ 777us) server_negotiation.cc:299] Negotiation successful
0807 18:13:03.531168 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":457,"thread_start_us":355,"threads_started":1}
I20260807 18:13:03.532612 27487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:03.511744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:40319 (local address 127.18.154.2:37867)
0807 18:13:03.512650 (+ 906us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:03.512692 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 18:13:03.514647 (+ 1955us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:13:03.515484 (+ 837us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:13:03.515494 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:13:03.516073 (+ 579us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:13:03.516726 (+ 653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:03.516746 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:03.518012 (+ 1266us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:03.518017 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:03.519426 (+ 1409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:03.519441 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:03.520391 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:03.520400 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:03.520680 (+ 280us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:03.521459 (+ 779us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:13:03.521491 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:13:03.524649 (+ 3158us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:13:03.528295 (+ 3646us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:13:03.528305 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:13:03.528330 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:13:03.528820 (+ 490us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:13:03.529270 (+ 450us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:13:03.529276 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:13:03.529279 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:13:03.529499 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:13:03.530117 (+ 618us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:13:03.530125 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:13:03.530538 (+ 413us) client_negotiation.cc:769] Sending connection context
0807 18:13:03.530830 (+ 292us) client_negotiation.cc:240] Negotiation successful
0807 18:13:03.531451 (+ 621us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":628,"thread_start_us":421,"threads_started":1}
I20260807 18:13:03.534250 27484 heartbeater.cc:347] Connected to a master server at 127.18.154.62:40319
I20260807 18:13:03.534575 27484 heartbeater.cc:464] Registering TS with master...
I20260807 18:13:03.535364 27484 heartbeater.cc:511] Master 127.18.154.62:40319 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.18.154.3@KRBTEST.COM; defaulting to no policy
I20260807 18:13:03.537062 27041 ts_manager.cc:194] Registered new tserver with Master: 5e1f7fd1245c43b9b1706676d2c1f0db (127.18.154.2:45645)
I20260807 18:13:03.537994 27041 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.154.2@KRBTEST.COM'} at 127.18.154.2:37867
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.154.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.154.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.3.keytab.
Entry for principal kudu/127.18.154.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.3.keytab.
I20260807 18:13:03.569698 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126328808105-19048-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126328808105-19048-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126328808105-19048-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126328808105-19048-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.154.3:0
--local_ip_for_outbound_sockets=127.18.154.3
--webserver_interface=127.18.154.3
--webserver_port=0
--tserver_master_addrs=127.18.154.62:40319
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.3.keytab
--principal=kudu/127.18.154.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.154.20:35225
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126328808105-19048-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126328808105-19048-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126328808105-19048-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:13:03.926898 27491 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:13:03.927212 27491 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:13:03.927383 27491 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:13:03.934909 27491 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:13:03.935052 27491 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:13:03.935256 27491 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.154.3
I20260807 18:13:03.945427 27491 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.154.20:35225
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126328808105-19048-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126328808105-19048-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.3.keytab
--principal=kudu/127.18.154.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.154.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126328808105-19048-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.154.3
--webserver_port=0
--tserver_master_addrs=127.18.154.62:40319
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.154.3
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126328808105-19048-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:13:03.946892 27491 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:13:03.948355 27491 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:13:03.963450 27507 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:13:03.963475 27503 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:13:03.963501 27502 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:13:03.967175 27491 server_base.cc:1034] running on GCE node
I20260807 18:13:03.967895 27491 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:13:03.968901 27491 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:13:03.970093 27491 hybrid_clock.cc:648] HybridClock initialized: now 1786126383970028 us; error 56 us; skew 500 ppm
I20260807 18:13:03.970413 27491 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:13:03 dist-test-slave-gkw7 krb5kdc[26979](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126383, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:13:03.973825 27491 init.cc:377] Logged in from keytab as kudu/127.18.154.3@KRBTEST.COM (short username kudu)
I20260807 18:13:03.975355 27491 webserver.cc:460] Webserver started at http://127.18.154.3:42769/ using document root <none> and password file <none>
I20260807 18:13:03.976109 27491 fs_manager.cc:359] Metadata directory not provided
I20260807 18:13:03.976250 27491 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:13:03.976586 27491 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:13:03.979678 27491 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126328808105-19048-0/minicluster-data/ts-2/data/instance:
uuid: "f86fd57c119340fcb63cf35f01b3739f"
format_stamp: "Formatted at 2026-08-07 18:13:03 on dist-test-slave-gkw7"
server_key: "59bd2231036d2ab48ec42a86776864a5"
server_key_iv: "b413f8b1b9ef4f47ec3e1dbde74e162c"
server_key_version: "encryptionkey@0"
I20260807 18:13:03.980651 27491 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126328808105-19048-0/minicluster-data/ts-2/wal/instance:
uuid: "f86fd57c119340fcb63cf35f01b3739f"
format_stamp: "Formatted at 2026-08-07 18:13:03 on dist-test-slave-gkw7"
server_key: "59bd2231036d2ab48ec42a86776864a5"
server_key_iv: "b413f8b1b9ef4f47ec3e1dbde74e162c"
server_key_version: "encryptionkey@0"
I20260807 18:13:03.986583 27491 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260807 18:13:03.987565 27299 heartbeater.cc:503] Master 127.18.154.62:40319 was elected leader, sending a full tablet report...
I20260807 18:13:03.990684 27516 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:13:03.992103 27491 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 18:13:03.992298 27491 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126328808105-19048-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126328808105-19048-0/minicluster-data/ts-2/wal
uuid: "f86fd57c119340fcb63cf35f01b3739f"
format_stamp: "Formatted at 2026-08-07 18:13:03 on dist-test-slave-gkw7"
server_key: "59bd2231036d2ab48ec42a86776864a5"
server_key_iv: "b413f8b1b9ef4f47ec3e1dbde74e162c"
server_key_version: "encryptionkey@0"
I20260807 18:13:03.992503 27491 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126328808105-19048-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126328808105-19048-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126328808105-19048-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:13:04.027449 27491 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:13:04.032298 27491 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:13:04.032648 27491 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:13:04.033880 27491 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:13:04.035422 27491 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:13:04.035509 27491 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:13:04.035718 27491 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:13:04.035778 27491 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:13:04.081887 27491 rpc_server.cc:307] RPC server started. Bound to: 127.18.154.3:42189
I20260807 18:13:04.081938 27659 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.154.3:42189 every 8 connection(s)
I20260807 18:13:04.083493 27491 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126328808105-19048-0/minicluster-data/ts-2/data/info.pb
I20260807 18:13:04.087462 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 27491
I20260807 18:13:04.087754 19048 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126328808105-19048-0/minicluster-data/ts-2/wal/instance
I20260807 18:13:04.088409 19048 external_mini_cluster.cc:1428] Setting key 7397081b2947009ea4ee00ac5d424e8f
Aug 07 18:13:04 dist-test-slave-gkw7 krb5kdc[26979](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126383, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.3@KRBTEST.COM for kudu/127.18.154.62@KRBTEST.COM
I20260807 18:13:04.106796 27664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:04.086551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.3:60403 (local address 127.18.154.62:40319)
0807 18:13:04.087032 (+ 481us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:04.087036 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:04.088317 (+ 1281us) server_negotiation.cc:407] Connection header received
0807 18:13:04.090570 (+ 2253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:04.090576 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:04.090677 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:04.090793 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:04.092467 (+ 1674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:04.093240 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:04.094962 (+ 1722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:04.095537 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:04.095675 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:04.100008 (+ 4333us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:13:04.100013 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:13:04.100016 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:13:04.100051 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:13:04.103313 (+ 3262us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:04.104365 (+ 1052us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:04.104371 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:04.104375 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:04.104555 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:04.105016 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:04.105019 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:04.105021 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:04.105304 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:13:04.105422 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0807 18:13:04.106249 (+ 827us) server_negotiation.cc:299] Negotiation successful
0807 18:13:04.106461 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":354,"thread_start_us":229,"threads_started":1}
I20260807 18:13:04.107785 27665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:04.087229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:40319 (local address 127.18.154.3:60403)
0807 18:13:04.088121 (+ 892us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:04.088200 (+ 79us) client_negotiation.cc:174] Beginning negotiation
0807 18:13:04.090240 (+ 2040us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:13:04.091048 (+ 808us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:13:04.091056 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:13:04.091715 (+ 659us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:13:04.092300 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:04.092319 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:04.093399 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:04.093403 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:04.094812 (+ 1409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:04.094825 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:04.095796 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:04.095805 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:04.096142 (+ 337us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:04.096988 (+ 846us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:13:04.097024 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:13:04.099780 (+ 2756us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:13:04.103494 (+ 3714us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:13:04.103503 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:13:04.103519 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:13:04.104158 (+ 639us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:13:04.104681 (+ 523us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:13:04.104684 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:13:04.104687 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:13:04.104900 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:13:04.105415 (+ 515us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:13:04.105421 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:13:04.105851 (+ 430us) client_negotiation.cc:769] Sending connection context
0807 18:13:04.106095 (+ 244us) client_negotiation.cc:240] Negotiation successful
0807 18:13:04.106535 (+ 440us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":655,"thread_start_us":429,"threads_started":1}
I20260807 18:13:04.109462 27660 heartbeater.cc:347] Connected to a master server at 127.18.154.62:40319
I20260807 18:13:04.109843 27660 heartbeater.cc:464] Registering TS with master...
I20260807 18:13:04.110605 27660 heartbeater.cc:511] Master 127.18.154.62:40319 requested a full tablet report, sending...
I20260807 18:13:04.112087 27041 ts_manager.cc:194] Registered new tserver with Master: f86fd57c119340fcb63cf35f01b3739f (127.18.154.3:42189)
I20260807 18:13:04.112937 27041 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.154.3@KRBTEST.COM'} at 127.18.154.3:60403
I20260807 18:13:04.117007 19048 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126328808105-19048-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126328808105-19048-0/krb5kdc/test-admin.keytab.
Aug 07 18:13:04 dist-test-slave-gkw7 krb5kdc[26979](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126384, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:13:04.152099 19048 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260807 18:13:04.158244 27673 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:13:04.155184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35590 (local address 127.18.154.1:43033)
0807 18:13:04.155994 (+ 810us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:04.156001 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:04.156030 (+ 29us) server_negotiation.cc:407] Connection header received
0807 18:13:04.156098 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:04.156105 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:04.156328 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:04.156515 (+ 187us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:04.157679 (+ 1164us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35590: BlockingRecv error: recv got EOF from 127.0.0.1:35590 (error 108)
Metrics: {"server-negotiator.queue_time_us":609,"thread_start_us":416,"threads_started":1}
W20260807 18:13:04.159317 19048 init.cc:244] Error while opening credential cache '/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126328808105-19048-0/krb5kdc/krb5cc' of type FILE: Runtime error: End of credential cache reached (filename: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126328808105-19048-0/krb5kdc/krb5cc)
Aug 07 18:13:04 dist-test-slave-gkw7 krb5kdc[26979](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126384, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:13:04.160935 19048 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 07 18:13:04 dist-test-slave-gkw7 krb5kdc[26979](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126384, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:13:04.162439 19048 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 07 18:13:04 dist-test-slave-gkw7 krb5kdc[26979](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126384, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.154.1@KRBTEST.COM
I20260807 18:13:04.178606 27673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:04.164169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35602 (local address 127.18.154.1:43033)
0807 18:13:04.164397 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:04.164404 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:04.164572 (+ 168us) server_negotiation.cc:407] Connection header received
0807 18:13:04.164726 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:04.164729 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:04.164794 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:04.164880 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:04.165979 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:04.166999 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:04.168409 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:04.168999 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:04.169132 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:04.172783 (+ 3651us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:13:04.172797 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:13:04.172813 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:13:04.172848 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:13:04.175821 (+ 2973us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:04.176510 (+ 689us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:04.176517 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:04.176519 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:04.176645 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:04.176946 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:04.176949 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:04.176951 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:04.177317 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:13:04.177609 (+ 292us) server_negotiation.cc:1035] Waiting for connection context
0807 18:13:04.178105 (+ 496us) server_negotiation.cc:299] Negotiation successful
0807 18:13:04.178319 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
W20260807 18:13:04.184952 27673 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:13:04.183206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35604 (local address 127.18.154.1:43033)
0807 18:13:04.183433 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:04.183439 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:04.183660 (+ 221us) server_negotiation.cc:407] Connection header received
0807 18:13:04.183821 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:04.183827 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:04.183896 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:04.183976 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:04.184681 (+ 705us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35604: BlockingRecv error: recv got EOF from 127.0.0.1:35604 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
W20260807 18:13:04.185091 19048 init.cc:244] Error while opening credential cache '/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126328808105-19048-0/krb5kdc/krb5cc' of type FILE: Runtime error: End of credential cache reached (filename: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126328808105-19048-0/krb5kdc/krb5cc)
Aug 07 18:13:04 dist-test-slave-gkw7 krb5kdc[26979](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126384, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:13:04.186792 19048 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 07 18:13:04 dist-test-slave-gkw7 krb5kdc[26979](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126384, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:13:04.188110 19048 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 07 18:13:04 dist-test-slave-gkw7 krb5kdc[26979](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126384, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.154.1@KRBTEST.COM
I20260807 18:13:04.202913 27673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:04.189437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35608 (local address 127.18.154.1:43033)
0807 18:13:04.189665 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:04.189671 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:04.189857 (+ 186us) server_negotiation.cc:407] Connection header received
0807 18:13:04.189986 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:04.189989 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:04.190045 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:04.190124 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:04.191098 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:04.191791 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:04.192975 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:04.193398 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:04.193492 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:04.197957 (+ 4465us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:13:04.197963 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:13:04.197966 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:13:04.198016 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:13:04.200509 (+ 2493us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:04.201159 (+ 650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:04.201163 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:04.201165 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:04.201389 (+ 224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:04.201792 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:04.201796 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:04.201798 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:04.202078 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:13:04.202310 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0807 18:13:04.202517 (+ 207us) server_negotiation.cc:299] Negotiation successful
0807 18:13:04.202682 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
W20260807 18:13:04.207736 27673 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:13:04.205740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35612 (local address 127.18.154.1:43033)
0807 18:13:04.205950 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:04.205957 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:04.206174 (+ 217us) server_negotiation.cc:407] Connection header received
0807 18:13:04.206371 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:04.206376 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:04.206461 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:04.206560 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:04.207469 (+ 909us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35612: BlockingRecv error: recv got EOF from 127.0.0.1:35612 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
W20260807 18:13:04.208027 19048 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786126328808105-19048-0/krb5kdc/krb5cc' of type FILE
Aug 07 18:13:04 dist-test-slave-gkw7 krb5kdc[26979](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126384, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:13:04.209755 19048 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 07 18:13:04 dist-test-slave-gkw7 krb5kdc[26979](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126384, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:13:04.211158 19048 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 07 18:13:04 dist-test-slave-gkw7 krb5kdc[26979](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126384, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.154.1@KRBTEST.COM
I20260807 18:13:04.227087 27673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:04.212712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35624 (local address 127.18.154.1:43033)
0807 18:13:04.213055 (+ 343us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:04.213062 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:04.213342 (+ 280us) server_negotiation.cc:407] Connection header received
0807 18:13:04.213593 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:04.213597 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:04.213666 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:04.213765 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:04.214701 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:04.215424 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:04.216675 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:04.217110 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:04.217243 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:04.221042 (+ 3799us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:13:04.221047 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:13:04.221051 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:13:04.221087 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:13:04.223997 (+ 2910us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:04.224764 (+ 767us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:04.224772 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:04.224776 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:04.224939 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:04.225364 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:04.225369 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:04.225371 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:04.225756 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:13:04.226108 (+ 352us) server_negotiation.cc:1035] Waiting for connection context
0807 18:13:04.226426 (+ 318us) server_negotiation.cc:299] Negotiation successful
0807 18:13:04.226679 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":179}
I20260807 18:13:04.228891 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 27121
I20260807 18:13:04.240964 27673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:04.229919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35632 (local address 127.18.154.1:43033)
0807 18:13:04.230095 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:04.230100 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:04.230184 (+ 84us) server_negotiation.cc:407] Connection header received
0807 18:13:04.230322 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:04.230326 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:04.230383 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:04.230469 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:04.231430 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:04.232156 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:04.233335 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:04.233840 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:04.233944 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:04.236000 (+ 2056us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:13:04.236006 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:13:04.236009 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:13:04.236042 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:13:04.238736 (+ 2694us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:04.239337 (+ 601us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:04.239340 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:04.239343 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:04.239471 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:04.239780 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:04.239783 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:04.239785 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:04.240047 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:13:04.240247 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0807 18:13:04.240515 (+ 268us) server_negotiation.cc:299] Negotiation successful
0807 18:13:04.240691 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260807 18:13:04.241679 27293 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:13:04.325968 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 27121
I20260807 18:13:04.351467 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 27312
Aug 07 18:13:04 dist-test-slave-gkw7 krb5kdc[26979](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126384, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.154.2@KRBTEST.COM
I20260807 18:13:04.367488 27690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:04.352340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53916 (local address 127.18.154.2:45645)
0807 18:13:04.353017 (+ 677us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:04.353024 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:04.353052 (+ 28us) server_negotiation.cc:407] Connection header received
0807 18:13:04.353127 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:04.353134 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:04.353352 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:04.353496 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:04.354516 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:04.355586 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:04.356786 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:04.357345 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:04.357441 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:04.361485 (+ 4044us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:13:04.361497 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:13:04.361512 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:13:04.361593 (+ 81us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:13:04.364628 (+ 3035us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:04.365254 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:04.365262 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:04.365264 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:04.365394 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:04.365794 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:04.365798 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:04.365800 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:04.366145 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:13:04.366381 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0807 18:13:04.366665 (+ 284us) server_negotiation.cc:299] Negotiation successful
0807 18:13:04.366975 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":528,"thread_start_us":373,"threads_started":1}
I20260807 18:13:04.369056 27477 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:13:04.452509 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 27312
I20260807 18:13:04.473233 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 27491
Aug 07 18:13:04 dist-test-slave-gkw7 krb5kdc[26979](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126384, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.154.3@KRBTEST.COM
I20260807 18:13:04.490604 27693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:04.474391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47962 (local address 127.18.154.3:42189)
0807 18:13:04.475082 (+ 691us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:04.475089 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:04.475119 (+ 30us) server_negotiation.cc:407] Connection header received
0807 18:13:04.475190 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:04.475197 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:04.475414 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:04.475599 (+ 185us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:04.476723 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:04.477960 (+ 1237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:04.479245 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:04.479825 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:04.479966 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:04.484159 (+ 4193us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:13:04.484172 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:13:04.484187 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:13:04.484231 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:13:04.487516 (+ 3285us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:04.488184 (+ 668us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:04.488191 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:04.488193 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:04.488393 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:04.488952 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:04.488956 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:04.488960 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:04.489498 (+ 538us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:13:04.489812 (+ 314us) server_negotiation.cc:1035] Waiting for connection context
0807 18:13:04.490110 (+ 298us) server_negotiation.cc:299] Negotiation successful
0807 18:13:04.490328 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":539,"thread_start_us":360,"threads_started":1}
I20260807 18:13:04.491952 27653 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:13:04.602348 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 27491
I20260807 18:13:04.625801 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 26994
I20260807 18:13:04.626987 27080 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:13:04.731938 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 26994
2026-08-07T18:13:04Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (4430 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:13:04 dist-test-slave-gkw7 krb5kdc[27700](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:13:04 dist-test-slave-gkw7 krb5kdc[27700](info): set up 2 sockets
Aug 07 18:13:04 dist-test-slave-gkw7 krb5kdc[27700](info): commencing operation
krb5kdc: starting...
W20260807 18:13:06.024528 19048 mini_kdc.cc:121] Time spent starting KDC: real 1.242s user 0.000s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/krb5kdc/test-user.keytab.
Aug 07 18:13:06 dist-test-slave-gkw7 krb5kdc[27700](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126386, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:13:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:13:06Z Disabled control of system clock
WARNING: no policy specified for oryx/127.18.154.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.154.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.154.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/krb5kdc/oryx_127.18.154.62.keytab.
Entry for principal oryx/127.18.154.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/krb5kdc/oryx_127.18.154.62.keytab.
I20260807 18:13:06.273205 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.154.62:34661
--webserver_interface=127.18.154.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/krb5kdc/oryx_127.18.154.62.keytab
--principal=oryx/127.18.154.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.154.20:38893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.154.62:34661
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:13:06.614879 27717 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:13:06.615201 27717 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:13:06.615355 27717 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:13:06.623193 27717 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:13:06.623342 27717 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:13:06.623431 27717 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:13:06.623529 27717 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:13:06.623608 27717 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:13:06.633688 27717 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.154.20:38893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.154.62:34661
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/krb5kdc/oryx_127.18.154.62.keytab
--principal=oryx/127.18.154.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.154.62:34661
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.154.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:13:06.635120 27717 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:13:06.636523 27717 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:13:06.650043 27722 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:13:06.650161 27726 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:13:06.650100 27723 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:13:06.651077 27717 server_base.cc:1034] running on GCE node
I20260807 18:13:06.651870 27717 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:13:06.653322 27717 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:13:06.654563 27717 hybrid_clock.cc:648] HybridClock initialized: now 1786126386654509 us; error 59 us; skew 500 ppm
I20260807 18:13:06.654927 27717 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:13:06 dist-test-slave-gkw7 krb5kdc[27700](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126386, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.154.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:13:06.658329 27717 init.cc:377] Logged in from keytab as oryx/127.18.154.62@KRBTEST.COM (short username oryx)
I20260807 18:13:06.659837 27717 webserver.cc:460] Webserver started at http://127.18.154.62:33637/ using document root <none> and password file <none>
I20260807 18:13:06.660532 27717 fs_manager.cc:359] Metadata directory not provided
I20260807 18:13:06.660636 27717 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:13:06.660897 27717 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:13:06.664160 27717 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/master-0/data/instance:
uuid: "cc97b2cc29884038b29e0e18de9a6d18"
format_stamp: "Formatted at 2026-08-07 18:13:06 on dist-test-slave-gkw7"
server_key: "00e587ac2eadd898ae88e8454f5f2599"
server_key_iv: "760aa4c755ce5c526b787b53955242e4"
server_key_version: "encryptionkey@0"
I20260807 18:13:06.664938 27717 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/master-0/wal/instance:
uuid: "cc97b2cc29884038b29e0e18de9a6d18"
format_stamp: "Formatted at 2026-08-07 18:13:06 on dist-test-slave-gkw7"
server_key: "00e587ac2eadd898ae88e8454f5f2599"
server_key_iv: "760aa4c755ce5c526b787b53955242e4"
server_key_version: "encryptionkey@0"
I20260807 18:13:06.670584 27717 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260807 18:13:06.674213 27739 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:13:06.675691 27717 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 18:13:06.675848 27717 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/master-0/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/master-0/wal
uuid: "cc97b2cc29884038b29e0e18de9a6d18"
format_stamp: "Formatted at 2026-08-07 18:13:06 on dist-test-slave-gkw7"
server_key: "00e587ac2eadd898ae88e8454f5f2599"
server_key_iv: "760aa4c755ce5c526b787b53955242e4"
server_key_version: "encryptionkey@0"
I20260807 18:13:06.676024 27717 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:13:06.695588 27717 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:13:06.700284 27717 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:13:06.700611 27717 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:13:06.720244 27717 rpc_server.cc:307] RPC server started. Bound to: 127.18.154.62:34661
I20260807 18:13:06.720257 27812 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.154.62:34661 every 8 connection(s)
I20260807 18:13:06.721805 27717 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
I20260807 18:13:06.726418 27813 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:13:06.727686 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 27717
I20260807 18:13:06.727950 19048 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/master-0/wal/instance
I20260807 18:13:06.728806 19048 external_mini_cluster.cc:1428] Setting key 2acfad860487f2b284a2c26f65750fb3
I20260807 18:13:06.742657 27813 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cc97b2cc29884038b29e0e18de9a6d18: Bootstrap starting.
Aug 07 18:13:06 dist-test-slave-gkw7 krb5kdc[27700](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126386, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.18.154.62@KRBTEST.COM
I20260807 18:13:06.747290 27813 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cc97b2cc29884038b29e0e18de9a6d18: Neither blocks nor log segments found. Creating new log.
I20260807 18:13:06.748987 27813 log.cc:824] T 00000000000000000000000000000000 P cc97b2cc29884038b29e0e18de9a6d18: Log is configured to *not* fsync() on all Append() calls
I20260807 18:13:06.752988 27813 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cc97b2cc29884038b29e0e18de9a6d18: No bootstrap required, opened a new log
I20260807 18:13:06.756381 27816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:06.731083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52272 (local address 127.18.154.62:34661)
0807 18:13:06.731997 (+ 914us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:06.732010 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:06.732060 (+ 50us) server_negotiation.cc:407] Connection header received
0807 18:13:06.733742 (+ 1682us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:06.733769 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:06.734240 (+ 471us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:06.734625 (+ 385us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:06.736672 (+ 2047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:06.738222 (+ 1550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:06.740152 (+ 1930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:06.740930 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:06.741085 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:06.746626 (+ 5541us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:13:06.746636 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:13:06.746655 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:13:06.746700 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:13:06.750887 (+ 4187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:06.751764 (+ 877us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:06.751775 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:06.751785 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:06.752028 (+ 243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:06.752636 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:06.752642 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:06.752645 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:06.753478 (+ 833us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:13:06.753899 (+ 421us) server_negotiation.cc:1035] Waiting for connection context
0807 18:13:06.754299 (+ 400us) server_negotiation.cc:299] Negotiation successful
0807 18:13:06.754812 (+ 513us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":685,"thread_start_us":393,"threads_started":1}
I20260807 18:13:06.762343 27813 raft_consensus.cc:348] T 00000000000000000000000000000000 P cc97b2cc29884038b29e0e18de9a6d18 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc97b2cc29884038b29e0e18de9a6d18" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 34661 } }
I20260807 18:13:06.762751 27813 raft_consensus.cc:374] T 00000000000000000000000000000000 P cc97b2cc29884038b29e0e18de9a6d18 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:13:06.762833 27813 raft_consensus.cc:729] T 00000000000000000000000000000000 P cc97b2cc29884038b29e0e18de9a6d18 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc97b2cc29884038b29e0e18de9a6d18, State: Initialized, Role: FOLLOWER
I20260807 18:13:06.763391 27813 consensus_queue.cc:260] T 00000000000000000000000000000000 P cc97b2cc29884038b29e0e18de9a6d18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc97b2cc29884038b29e0e18de9a6d18" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 34661 } }
I20260807 18:13:06.763605 27813 raft_consensus.cc:388] T 00000000000000000000000000000000 P cc97b2cc29884038b29e0e18de9a6d18 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:13:06.763715 27813 raft_consensus.cc:482] T 00000000000000000000000000000000 P cc97b2cc29884038b29e0e18de9a6d18 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:13:06.763854 27813 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cc97b2cc29884038b29e0e18de9a6d18 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:13:06.766520 27813 raft_consensus.cc:504] T 00000000000000000000000000000000 P cc97b2cc29884038b29e0e18de9a6d18 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc97b2cc29884038b29e0e18de9a6d18" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 34661 } }
I20260807 18:13:06.767040 27813 leader_election.cc:302] T 00000000000000000000000000000000 P cc97b2cc29884038b29e0e18de9a6d18 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cc97b2cc29884038b29e0e18de9a6d18; no voters:
I20260807 18:13:06.767613 27813 leader_election.cc:288] T 00000000000000000000000000000000 P cc97b2cc29884038b29e0e18de9a6d18 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:13:06.767824 27818 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cc97b2cc29884038b29e0e18de9a6d18 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:13:06.768580 27818 raft_consensus.cc:686] T 00000000000000000000000000000000 P cc97b2cc29884038b29e0e18de9a6d18 [term 1 LEADER]: Becoming Leader. State: Replica: cc97b2cc29884038b29e0e18de9a6d18, State: Running, Role: LEADER
I20260807 18:13:06.769181 27818 consensus_queue.cc:237] T 00000000000000000000000000000000 P cc97b2cc29884038b29e0e18de9a6d18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc97b2cc29884038b29e0e18de9a6d18" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 34661 } }
I20260807 18:13:06.769896 27813 sys_catalog.cc:564] T 00000000000000000000000000000000 P cc97b2cc29884038b29e0e18de9a6d18 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:13:06.772043 27824 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc97b2cc29884038b29e0e18de9a6d18 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cc97b2cc29884038b29e0e18de9a6d18. Latest consensus state: current_term: 1 leader_uuid: "cc97b2cc29884038b29e0e18de9a6d18" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc97b2cc29884038b29e0e18de9a6d18" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 34661 } } }
I20260807 18:13:06.772408 27824 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc97b2cc29884038b29e0e18de9a6d18 [sys.catalog]: This master's current role is: LEADER
I20260807 18:13:06.773566 27820 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc97b2cc29884038b29e0e18de9a6d18 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cc97b2cc29884038b29e0e18de9a6d18" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc97b2cc29884038b29e0e18de9a6d18" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 34661 } } }
I20260807 18:13:06.774057 27820 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc97b2cc29884038b29e0e18de9a6d18 [sys.catalog]: This master's current role is: LEADER
I20260807 18:13:06.775802 27830 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:13:06.782047 27830 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:13:06.791108 27830 catalog_manager.cc:1347] Generated new cluster ID: 6f80460f0d5e43d69126321a9a079ffa
I20260807 18:13:06.791245 27830 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:13:06.813326 27830 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:13:06.815454 27830 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:13:06.824395 27830 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cc97b2cc29884038b29e0e18de9a6d18: Generated new TSK 0
I20260807 18:13:06.825649 27830 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.18.154.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.154.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.154.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/krb5kdc/oryx_127.18.154.1.keytab.
Entry for principal oryx/127.18.154.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/krb5kdc/oryx_127.18.154.1.keytab.
I20260807 18:13:06.891263 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.154.1:0
--local_ip_for_outbound_sockets=127.18.154.1
--webserver_interface=127.18.154.1
--webserver_port=0
--tserver_master_addrs=127.18.154.62:34661
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/krb5kdc/oryx_127.18.154.1.keytab
--principal=oryx/127.18.154.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.154.20:38893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:13:07.246068 27856 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:13:07.246372 27856 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:13:07.246471 27856 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:13:07.253780 27856 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:13:07.253911 27856 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:13:07.254026 27856 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.154.1
I20260807 18:13:07.264226 27856 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.154.20:38893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/krb5kdc/oryx_127.18.154.1.keytab
--principal=oryx/127.18.154.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.154.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.154.1
--webserver_port=0
--tserver_master_addrs=127.18.154.62:34661
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.154.1
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:13:07.265635 27856 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:13:07.266973 27856 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:13:07.283546 27861 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:13:07.283582 27863 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:13:07.284408 27865 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:13:07.285686 27856 server_base.cc:1034] running on GCE node
I20260807 18:13:07.286374 27856 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:13:07.287480 27856 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:13:07.288727 27856 hybrid_clock.cc:648] HybridClock initialized: now 1786126387288665 us; error 52 us; skew 500 ppm
I20260807 18:13:07.289085 27856 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:13:07 dist-test-slave-gkw7 krb5kdc[27700](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126387, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.154.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:13:07.298171 27856 init.cc:377] Logged in from keytab as oryx/127.18.154.1@KRBTEST.COM (short username oryx)
I20260807 18:13:07.299782 27856 webserver.cc:460] Webserver started at http://127.18.154.1:37393/ using document root <none> and password file <none>
I20260807 18:13:07.300544 27856 fs_manager.cc:359] Metadata directory not provided
I20260807 18:13:07.300654 27856 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:13:07.300930 27856 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:13:07.304016 27856 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/ts-0/data/instance:
uuid: "3d0c878c4dd74021bd3406b93a11c7c4"
format_stamp: "Formatted at 2026-08-07 18:13:07 on dist-test-slave-gkw7"
server_key: "7e8cee6f20f17ebf0fad660de646409d"
server_key_iv: "35b8334b13fd23651f4c9e9af667bde6"
server_key_version: "encryptionkey@0"
I20260807 18:13:07.304788 27856 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/ts-0/wal/instance:
uuid: "3d0c878c4dd74021bd3406b93a11c7c4"
format_stamp: "Formatted at 2026-08-07 18:13:07 on dist-test-slave-gkw7"
server_key: "7e8cee6f20f17ebf0fad660de646409d"
server_key_iv: "35b8334b13fd23651f4c9e9af667bde6"
server_key_version: "encryptionkey@0"
I20260807 18:13:07.310297 27856 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260807 18:13:07.314087 27873 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:13:07.315596 27856 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 18:13:07.315763 27856 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/ts-0/wal
uuid: "3d0c878c4dd74021bd3406b93a11c7c4"
format_stamp: "Formatted at 2026-08-07 18:13:07 on dist-test-slave-gkw7"
server_key: "7e8cee6f20f17ebf0fad660de646409d"
server_key_iv: "35b8334b13fd23651f4c9e9af667bde6"
server_key_version: "encryptionkey@0"
I20260807 18:13:07.315951 27856 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:13:07.368121 27856 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:13:07.372627 27856 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:13:07.372900 27856 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:13:07.373924 27856 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:13:07.375308 27856 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:13:07.375389 27856 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:13:07.375485 27856 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:13:07.375533 27856 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:13:07.410965 27856 rpc_server.cc:307] RPC server started. Bound to: 127.18.154.1:41283
I20260807 18:13:07.411016 28026 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.154.1:41283 every 8 connection(s)
I20260807 18:13:07.412452 27856 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/ts-0/data/info.pb
I20260807 18:13:07.420775 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 27856
I20260807 18:13:07.421088 19048 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/ts-0/wal/instance
I20260807 18:13:07.421950 19048 external_mini_cluster.cc:1428] Setting key 54a6c4450adb549525874c27cc6c6ab7
Aug 07 18:13:07 dist-test-slave-gkw7 krb5kdc[27700](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126387, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.154.1@KRBTEST.COM for oryx/127.18.154.62@KRBTEST.COM
I20260807 18:13:07.436043 28029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:07.415331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.1:53831 (local address 127.18.154.62:34661)
0807 18:13:07.415861 (+ 530us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:07.415867 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:07.416927 (+ 1060us) server_negotiation.cc:407] Connection header received
0807 18:13:07.418885 (+ 1958us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:07.418889 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:07.418965 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:07.419074 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:07.420837 (+ 1763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:07.422098 (+ 1261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:07.424286 (+ 2188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:07.424754 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:07.424908 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:07.429433 (+ 4525us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:13:07.429440 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:13:07.429443 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:13:07.429481 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:13:07.432402 (+ 2921us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:07.433312 (+ 910us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:07.433320 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:07.433323 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:07.433481 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:07.433989 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:07.433994 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:07.433998 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:07.434298 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:13:07.434456 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0807 18:13:07.435254 (+ 798us) server_negotiation.cc:299] Negotiation successful
0807 18:13:07.435511 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":386,"thread_start_us":228,"threads_started":1}
I20260807 18:13:07.437290 28030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:07.415784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:34661 (local address 127.18.154.1:53831)
0807 18:13:07.416726 (+ 942us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:07.416777 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0807 18:13:07.418577 (+ 1800us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:13:07.419309 (+ 732us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:13:07.419317 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:13:07.419969 (+ 652us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:13:07.420577 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:07.420599 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:07.422317 (+ 1718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:07.422321 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:07.424096 (+ 1775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:07.424112 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:07.425039 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:07.425048 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:07.425610 (+ 562us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:07.426414 (+ 804us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:13:07.426441 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:13:07.429109 (+ 2668us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:13:07.432592 (+ 3483us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:13:07.432603 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:13:07.432625 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:13:07.433135 (+ 510us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:13:07.433633 (+ 498us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:13:07.433638 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:13:07.433642 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:13:07.433867 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:13:07.434446 (+ 579us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:13:07.434454 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:13:07.434845 (+ 391us) client_negotiation.cc:769] Sending connection context
0807 18:13:07.435134 (+ 289us) client_negotiation.cc:240] Negotiation successful
0807 18:13:07.435879 (+ 745us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":669,"thread_start_us":372,"threads_started":1}
I20260807 18:13:07.439373 28027 heartbeater.cc:347] Connected to a master server at 127.18.154.62:34661
I20260807 18:13:07.439828 28027 heartbeater.cc:464] Registering TS with master...
I20260807 18:13:07.441092 28027 heartbeater.cc:511] Master 127.18.154.62:34661 requested a full tablet report, sending...
I20260807 18:13:07.444033 27766 ts_manager.cc:194] Registered new tserver with Master: 3d0c878c4dd74021bd3406b93a11c7c4 (127.18.154.1:41283)
I20260807 18:13:07.446627 27766 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.18.154.1@KRBTEST.COM'} at 127.18.154.1:53831
WARNING: no policy specified for oryx/127.18.154.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.154.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.154.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/krb5kdc/oryx_127.18.154.2.keytab.
Entry for principal oryx/127.18.154.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/krb5kdc/oryx_127.18.154.2.keytab.
I20260807 18:13:07.489045 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.154.2:0
--local_ip_for_outbound_sockets=127.18.154.2
--webserver_interface=127.18.154.2
--webserver_port=0
--tserver_master_addrs=127.18.154.62:34661
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/krb5kdc/oryx_127.18.154.2.keytab
--principal=oryx/127.18.154.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.154.20:38893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:13:07.837632 28036 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:13:07.837957 28036 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:13:07.838059 28036 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:13:07.845630 28036 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:13:07.845768 28036 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:13:07.845950 28036 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.154.2
I20260807 18:13:07.856127 28036 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.154.20:38893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/krb5kdc/oryx_127.18.154.2.keytab
--principal=oryx/127.18.154.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.154.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.154.2
--webserver_port=0
--tserver_master_addrs=127.18.154.62:34661
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.154.2
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:13:07.857576 28036 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:13:07.858979 28036 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:13:07.873114 28044 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:13:07.873135 28045 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:13:07.873332 28047 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:13:07.874045 28036 server_base.cc:1034] running on GCE node
I20260807 18:13:07.874629 28036 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:13:07.875573 28036 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:13:07.876819 28036 hybrid_clock.cc:648] HybridClock initialized: now 1786126387876751 us; error 66 us; skew 500 ppm
I20260807 18:13:07.877197 28036 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:13:07 dist-test-slave-gkw7 krb5kdc[27700](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126387, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.154.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:13:07.880987 28036 init.cc:377] Logged in from keytab as oryx/127.18.154.2@KRBTEST.COM (short username oryx)
I20260807 18:13:07.882651 28036 webserver.cc:460] Webserver started at http://127.18.154.2:34707/ using document root <none> and password file <none>
I20260807 18:13:07.883379 28036 fs_manager.cc:359] Metadata directory not provided
I20260807 18:13:07.883493 28036 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:13:07.883786 28036 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:13:07.887267 28036 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/ts-1/data/instance:
uuid: "fde1306083f041638796f175e031fb30"
format_stamp: "Formatted at 2026-08-07 18:13:07 on dist-test-slave-gkw7"
server_key: "ddd3048d1aacb8b1960999022f2ccca5"
server_key_iv: "f67d048bda83e2248ced505d2aa5adeb"
server_key_version: "encryptionkey@0"
I20260807 18:13:07.888098 28036 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/ts-1/wal/instance:
uuid: "fde1306083f041638796f175e031fb30"
format_stamp: "Formatted at 2026-08-07 18:13:07 on dist-test-slave-gkw7"
server_key: "ddd3048d1aacb8b1960999022f2ccca5"
server_key_iv: "f67d048bda83e2248ced505d2aa5adeb"
server_key_version: "encryptionkey@0"
I20260807 18:13:07.893951 28036 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.003s
I20260807 18:13:07.897806 28060 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:13:07.899281 28036 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 18:13:07.899433 28036 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/ts-1/wal
uuid: "fde1306083f041638796f175e031fb30"
format_stamp: "Formatted at 2026-08-07 18:13:07 on dist-test-slave-gkw7"
server_key: "ddd3048d1aacb8b1960999022f2ccca5"
server_key_iv: "f67d048bda83e2248ced505d2aa5adeb"
server_key_version: "encryptionkey@0"
I20260807 18:13:07.899618 28036 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:13:07.931849 28036 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:13:07.936982 28036 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:13:07.937337 28036 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:13:07.938473 28036 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:13:07.939936 28036 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:13:07.940047 28036 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:13:07.940207 28036 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:13:07.940299 28036 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:13:07.975921 28036 rpc_server.cc:307] RPC server started. Bound to: 127.18.154.2:42923
I20260807 18:13:07.975982 28220 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.154.2:42923 every 8 connection(s)
I20260807 18:13:07.977366 28036 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/ts-1/data/info.pb
I20260807 18:13:07.986690 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 28036
I20260807 18:13:07.986938 19048 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/ts-1/wal/instance
I20260807 18:13:07.987385 19048 external_mini_cluster.cc:1428] Setting key f7f92ea73086929bbc23b3280506e68f
Aug 07 18:13:07 dist-test-slave-gkw7 krb5kdc[27700](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126387, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.154.2@KRBTEST.COM for oryx/127.18.154.62@KRBTEST.COM
I20260807 18:13:08.004185 28223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:07.980169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.2:58929 (local address 127.18.154.62:34661)
0807 18:13:07.980676 (+ 507us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:07.980680 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:07.981768 (+ 1088us) server_negotiation.cc:407] Connection header received
0807 18:13:07.983622 (+ 1854us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:07.983628 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:07.983729 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:07.983854 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:07.985392 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:07.986200 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:07.987931 (+ 1731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:07.988408 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:07.988546 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:07.997038 (+ 8492us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:13:07.997045 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:13:07.997049 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:13:07.997086 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:13:08.000335 (+ 3249us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:08.001359 (+ 1024us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:08.001364 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:08.001366 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:08.001510 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:08.002104 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:08.002107 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:08.002109 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:08.002434 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:13:08.002568 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0807 18:13:08.003634 (+ 1066us) server_negotiation.cc:299] Negotiation successful
0807 18:13:08.003821 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":397,"thread_start_us":263,"threads_started":1}
I20260807 18:13:08.005944 28225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:07.980771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:34661 (local address 127.18.154.2:58929)
0807 18:13:07.981591 (+ 820us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:07.981646 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0807 18:13:07.983344 (+ 1698us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:13:07.984101 (+ 757us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:13:07.984108 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:13:07.984627 (+ 519us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:13:07.985203 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:07.985230 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:07.986362 (+ 1132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:07.986366 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:07.987787 (+ 1421us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:07.987800 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:07.988622 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:07.988627 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:07.988863 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:07.993776 (+ 4913us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:13:07.993830 (+ 54us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:13:07.996756 (+ 2926us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:13:08.000506 (+ 3750us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:13:08.000519 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:13:08.000542 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:13:08.001201 (+ 659us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:13:08.001686 (+ 485us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:13:08.001694 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:13:08.001700 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:13:08.001986 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:13:08.002582 (+ 596us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:13:08.002594 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:13:08.003121 (+ 527us) client_negotiation.cc:769] Sending connection context
0807 18:13:08.003506 (+ 385us) client_negotiation.cc:240] Negotiation successful
0807 18:13:08.004144 (+ 638us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":530,"thread_start_us":350,"threads_started":1}
I20260807 18:13:08.007599 28221 heartbeater.cc:347] Connected to a master server at 127.18.154.62:34661
I20260807 18:13:08.007946 28221 heartbeater.cc:464] Registering TS with master...
I20260807 18:13:08.008772 28221 heartbeater.cc:511] Master 127.18.154.62:34661 requested a full tablet report, sending...
I20260807 18:13:08.010454 27766 ts_manager.cc:194] Registered new tserver with Master: fde1306083f041638796f175e031fb30 (127.18.154.2:42923)
I20260807 18:13:08.011332 27766 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.18.154.2@KRBTEST.COM'} at 127.18.154.2:58929
WARNING: no policy specified for oryx/127.18.154.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.154.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.154.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/krb5kdc/oryx_127.18.154.3.keytab.
Entry for principal oryx/127.18.154.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/krb5kdc/oryx_127.18.154.3.keytab.
I20260807 18:13:08.049985 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.154.3:0
--local_ip_for_outbound_sockets=127.18.154.3
--webserver_interface=127.18.154.3
--webserver_port=0
--tserver_master_addrs=127.18.154.62:34661
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/krb5kdc/oryx_127.18.154.3.keytab
--principal=oryx/127.18.154.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.154.20:38893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:13:08.405794 28228 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:13:08.406179 28228 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:13:08.406343 28228 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:13:08.413902 28228 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:13:08.414057 28228 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:13:08.414242 28228 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.154.3
I20260807 18:13:08.424403 28228 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.154.20:38893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/krb5kdc/oryx_127.18.154.3.keytab
--principal=oryx/127.18.154.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.154.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.154.3
--webserver_port=0
--tserver_master_addrs=127.18.154.62:34661
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.154.3
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:13:08.425873 28228 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:13:08.427292 28228 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:13:08.441560 28236 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:13:08.441632 28238 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:13:08.441591 28235 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:13:08.442647 28228 server_base.cc:1034] running on GCE node
I20260807 18:13:08.443351 28228 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:13:08.444415 28228 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:13:08.445659 28228 hybrid_clock.cc:648] HybridClock initialized: now 1786126388445610 us; error 57 us; skew 500 ppm
I20260807 18:13:08.446013 28228 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:13:08 dist-test-slave-gkw7 krb5kdc[27700](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126388, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.154.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:13:08.449431 28228 init.cc:377] Logged in from keytab as oryx/127.18.154.3@KRBTEST.COM (short username oryx)
I20260807 18:13:08.451002 28027 heartbeater.cc:503] Master 127.18.154.62:34661 was elected leader, sending a full tablet report...
I20260807 18:13:08.451195 28228 webserver.cc:460] Webserver started at http://127.18.154.3:38257/ using document root <none> and password file <none>
I20260807 18:13:08.452033 28228 fs_manager.cc:359] Metadata directory not provided
I20260807 18:13:08.452188 28228 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:13:08.452515 28228 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:13:08.455807 28228 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/ts-2/data/instance:
uuid: "9b53136e1bd4469caee5c7f30bad8a38"
format_stamp: "Formatted at 2026-08-07 18:13:08 on dist-test-slave-gkw7"
server_key: "af387737d31f90b287d05e24f831e769"
server_key_iv: "eb906094aa4dfe59679b4a87edd7d701"
server_key_version: "encryptionkey@0"
I20260807 18:13:08.456727 28228 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/ts-2/wal/instance:
uuid: "9b53136e1bd4469caee5c7f30bad8a38"
format_stamp: "Formatted at 2026-08-07 18:13:08 on dist-test-slave-gkw7"
server_key: "af387737d31f90b287d05e24f831e769"
server_key_iv: "eb906094aa4dfe59679b4a87edd7d701"
server_key_version: "encryptionkey@0"
I20260807 18:13:08.462657 28228 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260807 18:13:08.466434 28247 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:13:08.467882 28228 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 18:13:08.468077 28228 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/ts-2/wal
uuid: "9b53136e1bd4469caee5c7f30bad8a38"
format_stamp: "Formatted at 2026-08-07 18:13:08 on dist-test-slave-gkw7"
server_key: "af387737d31f90b287d05e24f831e769"
server_key_iv: "eb906094aa4dfe59679b4a87edd7d701"
server_key_version: "encryptionkey@0"
I20260807 18:13:08.468302 28228 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:13:08.493441 28228 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:13:08.497921 28228 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:13:08.498255 28228 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:13:08.499353 28228 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:13:08.500777 28228 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:13:08.500886 28228 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:13:08.501037 28228 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:13:08.501114 28228 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:13:08.536391 28228 rpc_server.cc:307] RPC server started. Bound to: 127.18.154.3:46263
I20260807 18:13:08.536449 28403 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.154.3:46263 every 8 connection(s)
I20260807 18:13:08.538188 28228 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/ts-2/data/info.pb
I20260807 18:13:08.540129 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 28228
I20260807 18:13:08.540410 19048 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/ts-2/wal/instance
I20260807 18:13:08.541016 19048 external_mini_cluster.cc:1428] Setting key 85125d1df935ba98adfa740ed21bcd43
Aug 07 18:13:08 dist-test-slave-gkw7 krb5kdc[27700](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126388, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.154.3@KRBTEST.COM for oryx/127.18.154.62@KRBTEST.COM
I20260807 18:13:08.559553 28407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:08.541257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.3:38259 (local address 127.18.154.62:34661)
0807 18:13:08.542013 (+ 756us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:08.542018 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:08.542946 (+ 928us) server_negotiation.cc:407] Connection header received
0807 18:13:08.545093 (+ 2147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:08.545099 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:08.545187 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:08.545317 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:08.546923 (+ 1606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:08.547711 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:08.549259 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:08.549746 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:08.549878 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:08.553479 (+ 3601us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:13:08.553484 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:13:08.553487 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:13:08.553520 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:13:08.556325 (+ 2805us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:08.557121 (+ 796us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:08.557128 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:08.557132 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:08.557259 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:08.557765 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:08.557770 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:08.557774 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:08.558052 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:13:08.558230 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0807 18:13:08.558939 (+ 709us) server_negotiation.cc:299] Negotiation successful
0807 18:13:08.559163 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":612,"thread_start_us":466,"threads_started":1}
I20260807 18:13:08.560562 28408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:08.541999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:34661 (local address 127.18.154.3:38259)
0807 18:13:08.542787 (+ 788us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:08.542837 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0807 18:13:08.544830 (+ 1993us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:13:08.545631 (+ 801us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:13:08.545644 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:13:08.546180 (+ 536us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:13:08.546756 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:08.546773 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:08.547869 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:08.547879 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:08.549108 (+ 1229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:08.549120 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:08.549971 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:08.549979 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:08.550218 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:08.550879 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:13:08.550901 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:13:08.553263 (+ 2362us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:13:08.556473 (+ 3210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:13:08.556484 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:13:08.556501 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:13:08.557000 (+ 499us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:13:08.557376 (+ 376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:13:08.557381 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:13:08.557385 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:13:08.557652 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:13:08.558194 (+ 542us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:13:08.558204 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:13:08.558561 (+ 357us) client_negotiation.cc:769] Sending connection context
0807 18:13:08.558829 (+ 268us) client_negotiation.cc:240] Negotiation successful
0807 18:13:08.559450 (+ 621us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":533,"thread_start_us":325,"threads_started":1}
I20260807 18:13:08.562041 28405 heartbeater.cc:347] Connected to a master server at 127.18.154.62:34661
I20260807 18:13:08.562359 28405 heartbeater.cc:464] Registering TS with master...
I20260807 18:13:08.563102 28405 heartbeater.cc:511] Master 127.18.154.62:34661 requested a full tablet report, sending...
I20260807 18:13:08.564637 27766 ts_manager.cc:194] Registered new tserver with Master: 9b53136e1bd4469caee5c7f30bad8a38 (127.18.154.3:46263)
I20260807 18:13:08.565510 27766 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.18.154.3@KRBTEST.COM'} at 127.18.154.3:38259
I20260807 18:13:08.569598 19048 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.18.154.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.154.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.154.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/krb5kdc/oryx_127.18.154.61.keytab.
Entry for principal oryx/127.18.154.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/krb5kdc/oryx_127.18.154.61.keytab.
I20260807 18:13:08.629658 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.154.61:38131
--webserver_interface=127.18.154.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/krb5kdc/oryx_127.18.154.61.keytab
--principal=oryx/127.18.154.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.154.20:38893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.154.62:34661,127.18.154.61:38131
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:13:08.975149 28411 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:13:08.975492 28411 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:13:08.975643 28411 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:13:08.983422 28411 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:13:08.983600 28411 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:13:08.983665 28411 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:13:08.983717 28411 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:13:08.983762 28411 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:13:08.993865 28411 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.154.20:38893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.18.154.62:34661,127.18.154.61:38131
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/krb5kdc/oryx_127.18.154.61.keytab
--principal=oryx/127.18.154.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.154.61:38131
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.18.154.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
W20260807 18:13:08.994204 28411 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.18.154.62:34661,127.18.154.61:38131'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260807 18:13:08.995237 28411 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:13:08.996619 28411 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:13:09.009152 28419 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:13:09.009246 28420 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:13:09.009262 28422 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:13:09.010185 28411 server_base.cc:1034] running on GCE node
I20260807 18:13:09.010991 28411 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:13:09.012466 28411 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:13:09.013705 28411 hybrid_clock.cc:648] HybridClock initialized: now 1786126389013627 us; error 51 us; skew 500 ppm
I20260807 18:13:09.014140 28411 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 18:13:09.014209 28221 heartbeater.cc:503] Master 127.18.154.62:34661 was elected leader, sending a full tablet report...
Aug 07 18:13:09 dist-test-slave-gkw7 krb5kdc[27700](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126389, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.154.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:13:09.017889 28411 init.cc:377] Logged in from keytab as oryx/127.18.154.61@KRBTEST.COM (short username oryx)
I20260807 18:13:09.019954 28411 webserver.cc:460] Webserver started at http://127.18.154.61:45741/ using document root <none> and password file <none>
I20260807 18:13:09.020820 28411 fs_manager.cc:359] Metadata directory not provided
I20260807 18:13:09.020972 28411 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:13:09.021347 28411 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:13:09.024731 28411 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/master-1/data/instance:
uuid: "a6983037814748628555659bcf04b165"
format_stamp: "Formatted at 2026-08-07 18:13:09 on dist-test-slave-gkw7"
server_key: "a88e101c16b9b6cf938cbe366e1885c9"
server_key_iv: "f57e24ed41fe939be511907bf4819990"
server_key_version: "encryptionkey@0"
I20260807 18:13:09.025619 28411 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/master-1/wal/instance:
uuid: "a6983037814748628555659bcf04b165"
format_stamp: "Formatted at 2026-08-07 18:13:09 on dist-test-slave-gkw7"
server_key: "a88e101c16b9b6cf938cbe366e1885c9"
server_key_iv: "f57e24ed41fe939be511907bf4819990"
server_key_version: "encryptionkey@0"
I20260807 18:13:09.031198 28411 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260807 18:13:09.035024 28433 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:13:09.036612 28411 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 18:13:09.036808 28411 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/master-1/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/master-1/wal
uuid: "a6983037814748628555659bcf04b165"
format_stamp: "Formatted at 2026-08-07 18:13:09 on dist-test-slave-gkw7"
server_key: "a88e101c16b9b6cf938cbe366e1885c9"
server_key_iv: "f57e24ed41fe939be511907bf4819990"
server_key_version: "encryptionkey@0"
I20260807 18:13:09.037051 28411 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:13:09.077118 28411 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:13:09.082193 28411 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:13:09.082479 28411 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:13:09.102589 28411 rpc_server.cc:307] RPC server started. Bound to: 127.18.154.61:38131
I20260807 18:13:09.102635 28502 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.154.61:38131 every 8 connection(s)
I20260807 18:13:09.104256 28411 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/master-1/data/info.pb
I20260807 18:13:09.107059 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 28411
I20260807 18:13:09.109027 28503 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:13:09.119088 28503 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.154.62" port: 34661 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 18:13:09 dist-test-slave-gkw7 krb5kdc[27700](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126386, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.18.154.61@KRBTEST.COM
Aug 07 18:13:09 dist-test-slave-gkw7 krb5kdc[27700](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126389, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.154.61@KRBTEST.COM for oryx/127.18.154.62@KRBTEST.COM
I20260807 18:13:09.133769 28507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:09.109513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52194 (local address 127.18.154.61:38131)
0807 18:13:09.110361 (+ 848us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:09.110373 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:09.110413 (+ 40us) server_negotiation.cc:407] Connection header received
0807 18:13:09.111977 (+ 1564us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:09.111998 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:09.112401 (+ 403us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:09.112745 (+ 344us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:09.114757 (+ 2012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:09.117461 (+ 2704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:09.118889 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:09.119722 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:09.119942 (+ 220us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:09.125276 (+ 5334us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:13:09.125284 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:13:09.125299 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:13:09.125338 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:13:09.128915 (+ 3577us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:09.129862 (+ 947us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:09.129872 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:09.129885 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:09.130088 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:09.130542 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:09.130545 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:09.130547 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:09.131078 (+ 531us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:13:09.131343 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0807 18:13:09.131930 (+ 587us) server_negotiation.cc:299] Negotiation successful
0807 18:13:09.132399 (+ 469us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":601,"thread_start_us":338,"threads_started":1}
I20260807 18:13:09.140749 28508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:09.122105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52282 (local address 127.18.154.62:34661)
0807 18:13:09.123791 (+ 1686us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:09.123795 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:09.123811 (+ 16us) server_negotiation.cc:407] Connection header received
0807 18:13:09.123864 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:09.123867 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:09.123934 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:09.124023 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:09.125639 (+ 1616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:09.126870 (+ 1231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:09.128307 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:09.128968 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:09.129088 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:09.134762 (+ 5674us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:13:09.134770 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:13:09.134773 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:13:09.134812 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:13:09.137863 (+ 3051us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:09.138593 (+ 730us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:09.138598 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:09.138604 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:09.138734 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:09.139121 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:09.139127 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:09.139130 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:09.139478 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:13:09.139642 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0807 18:13:09.140130 (+ 488us) server_negotiation.cc:299] Negotiation successful
0807 18:13:09.140382 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1584,"thread_start_us":1469,"threads_started":1}
I20260807 18:13:09.140817 28509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:09.122086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:34661 (local address 127.0.0.1:52282)
0807 18:13:09.123031 (+ 945us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:09.123103 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0807 18:13:09.123327 (+ 224us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:13:09.124164 (+ 837us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:13:09.124172 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:13:09.125088 (+ 916us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:13:09.125449 (+ 361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:09.125459 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:09.127023 (+ 1564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:09.127027 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:09.128155 (+ 1128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:09.128167 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:09.129160 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:09.129167 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:09.129382 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:09.130175 (+ 793us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:13:09.130204 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:13:09.134564 (+ 4360us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:13:09.138017 (+ 3453us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:13:09.138028 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:13:09.138036 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:13:09.138455 (+ 419us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:13:09.138852 (+ 397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:13:09.138857 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:13:09.138861 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:13:09.139008 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:13:09.139617 (+ 609us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:13:09.139625 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:13:09.139816 (+ 191us) client_negotiation.cc:769] Sending connection context
0807 18:13:09.140031 (+ 215us) client_negotiation.cc:240] Negotiation successful
0807 18:13:09.140374 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":583,"spinlock_wait_cycles":17152,"thread_start_us":370,"threads_started":1}
I20260807 18:13:09.142838 28503 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.154.61" port: 38131 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 18:13:09 dist-test-slave-gkw7 krb5kdc[27700](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126389, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.154.61@KRBTEST.COM for oryx/127.18.154.61@KRBTEST.COM
I20260807 18:13:09.155669 28509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:09.143671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.61:38131 (local address 127.0.0.1:52198)
0807 18:13:09.143830 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:09.143851 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 18:13:09.143989 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:13:09.144283 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:13:09.144287 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:13:09.144646 (+ 359us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:13:09.144779 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:09.144788 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:09.145873 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:09.145878 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:09.146726 (+ 848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:09.146737 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:09.147443 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:09.147449 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:09.147643 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:09.148230 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:13:09.148246 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:13:09.150334 (+ 2088us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:13:09.153212 (+ 2878us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:13:09.153216 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:13:09.153220 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:13:09.153667 (+ 447us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:13:09.154033 (+ 366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:13:09.154039 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:13:09.154043 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:13:09.154182 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:13:09.154686 (+ 504us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:13:09.154690 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:13:09.154839 (+ 149us) client_negotiation.cc:769] Sending connection context
0807 18:13:09.154981 (+ 142us) client_negotiation.cc:240] Negotiation successful
0807 18:13:09.155286 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260807 18:13:09.155774 28507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:09.143618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52198 (local address 127.18.154.61:38131)
0807 18:13:09.143831 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:09.143838 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:09.143926 (+ 88us) server_negotiation.cc:407] Connection header received
0807 18:13:09.144093 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:09.144098 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:09.144160 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:09.144258 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:09.144919 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:09.145727 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:09.146866 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:09.147292 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:09.147418 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:09.150513 (+ 3095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:13:09.150518 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:13:09.150521 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:13:09.150558 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:13:09.153065 (+ 2507us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:09.153786 (+ 721us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:09.153791 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:09.153795 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:09.153925 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:09.154284 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:09.154291 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:09.154297 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:09.154570 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:13:09.154723 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0807 18:13:09.155056 (+ 333us) server_negotiation.cc:299] Negotiation successful
0807 18:13:09.155430 (+ 374us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":158,"server-negotiator.queue_time_us":60}
I20260807 18:13:09.160924 28503 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a6983037814748628555659bcf04b165: Bootstrap starting.
I20260807 18:13:09.164145 28503 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a6983037814748628555659bcf04b165: Neither blocks nor log segments found. Creating new log.
I20260807 18:13:09.165225 28503 log.cc:824] T 00000000000000000000000000000000 P a6983037814748628555659bcf04b165: Log is configured to *not* fsync() on all Append() calls
I20260807 18:13:09.167932 28503 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a6983037814748628555659bcf04b165: No bootstrap required, opened a new log
I20260807 18:13:09.175478 28503 raft_consensus.cc:348] T 00000000000000000000000000000000 P a6983037814748628555659bcf04b165 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc97b2cc29884038b29e0e18de9a6d18" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 34661 } } peers { permanent_uuid: "a6983037814748628555659bcf04b165" member_type: VOTER last_known_addr { host: "127.18.154.61" port: 38131 } }
I20260807 18:13:09.175838 28503 raft_consensus.cc:374] T 00000000000000000000000000000000 P a6983037814748628555659bcf04b165 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:13:09.175947 28503 raft_consensus.cc:729] T 00000000000000000000000000000000 P a6983037814748628555659bcf04b165 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6983037814748628555659bcf04b165, State: Initialized, Role: FOLLOWER
I20260807 18:13:09.176676 28503 consensus_queue.cc:260] T 00000000000000000000000000000000 P a6983037814748628555659bcf04b165 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc97b2cc29884038b29e0e18de9a6d18" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 34661 } } peers { permanent_uuid: "a6983037814748628555659bcf04b165" member_type: VOTER last_known_addr { host: "127.18.154.61" port: 38131 } }
I20260807 18:13:09.177620 28511 sys_catalog.cc:455] T 00000000000000000000000000000000 P a6983037814748628555659bcf04b165 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc97b2cc29884038b29e0e18de9a6d18" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 34661 } } peers { permanent_uuid: "a6983037814748628555659bcf04b165" member_type: VOTER last_known_addr { host: "127.18.154.61" port: 38131 } } }
I20260807 18:13:09.177989 28511 sys_catalog.cc:458] T 00000000000000000000000000000000 P a6983037814748628555659bcf04b165 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 18:13:09.178670 28503 sys_catalog.cc:564] T 00000000000000000000000000000000 P a6983037814748628555659bcf04b165 [sys.catalog]: configured and running, proceeding with master startup.
W20260807 18:13:09.188818 28523 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a6983037814748628555659bcf04b165: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 18:13:09.188989 28523 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 18:13:09.192104 28411 master_runner.cc:428] Detected that this master a6983037814748628555659bcf04b165 is joining an existing cluster
I20260807 18:13:09.192216 28411 master_runner.cc:432] Initiating AddMaster RPC to add 127.18.154.61:38131
I20260807 18:13:09.216698 28508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:09.197615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52292 (local address 127.18.154.62:34661)
0807 18:13:09.197824 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:09.197831 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:09.197850 (+ 19us) server_negotiation.cc:407] Connection header received
0807 18:13:09.197913 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:09.197919 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:09.198021 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:09.198124 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:09.200323 (+ 2199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:09.201826 (+ 1503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:09.203797 (+ 1971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:09.204246 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:09.204407 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:09.207203 (+ 2796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:13:09.207210 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:13:09.207215 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:13:09.207258 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:13:09.211156 (+ 3898us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:09.212038 (+ 882us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:09.212044 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:09.212047 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:09.212209 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:09.212676 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:09.212682 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:09.212687 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:09.213138 (+ 451us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:13:09.213283 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0807 18:13:09.216035 (+ 2752us) server_negotiation.cc:299] Negotiation successful
0807 18:13:09.216298 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260807 18:13:09.217859 28531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:09.196430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:34661 (local address 127.0.0.1:52292)
0807 18:13:09.197326 (+ 896us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:09.197349 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 18:13:09.197497 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:13:09.198341 (+ 844us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:13:09.198349 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:13:09.199504 (+ 1155us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:13:09.200085 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:09.200103 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:09.201995 (+ 1892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:09.201999 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:09.203652 (+ 1653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:09.203665 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:09.204448 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:09.204454 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:09.204670 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:09.205385 (+ 715us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:13:09.205407 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:13:09.206974 (+ 1567us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:13:09.211322 (+ 4348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:13:09.211327 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:13:09.211331 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:13:09.211872 (+ 541us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:13:09.212373 (+ 501us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:13:09.212377 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:13:09.212381 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:13:09.212539 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:13:09.213403 (+ 864us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:13:09.213413 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:13:09.214400 (+ 987us) client_negotiation.cc:769] Sending connection context
0807 18:13:09.216947 (+ 2547us) client_negotiation.cc:240] Negotiation successful
0807 18:13:09.217312 (+ 365us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":518,"mutex_wait_us":3,"thread_start_us":1646,"threads_started":1}
I20260807 18:13:09.219534 28507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:09.197891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52204 (local address 127.18.154.61:38131)
0807 18:13:09.198065 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:09.198073 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:09.202220 (+ 4147us) server_negotiation.cc:407] Connection header received
0807 18:13:09.202471 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:09.202478 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:09.202584 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:09.202717 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:09.204498 (+ 1781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:09.206047 (+ 1549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:09.208746 (+ 2699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:09.209846 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:09.209959 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:09.212430 (+ 2471us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:13:09.212438 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:13:09.212445 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:13:09.212477 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:13:09.215560 (+ 3083us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:09.216403 (+ 843us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:09.216408 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:09.216412 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:09.216577 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:09.217438 (+ 861us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:09.217444 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:09.217447 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:09.217823 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:13:09.218431 (+ 608us) server_negotiation.cc:1035] Waiting for connection context
0807 18:13:09.218934 (+ 503us) server_negotiation.cc:299] Negotiation successful
0807 18:13:09.219150 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":28,"server-negotiator.queue_time_us":55}
I20260807 18:13:09.220764 28532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:09.197184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.61:38131 (local address 127.0.0.1:52204)
0807 18:13:09.202085 (+ 4901us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:09.202113 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0807 18:13:09.202294 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:13:09.203002 (+ 708us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:13:09.203009 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:13:09.203962 (+ 953us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:13:09.204333 (+ 371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:09.204356 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:09.206251 (+ 1895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:09.206255 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:09.208600 (+ 2345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:09.208617 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:09.210275 (+ 1658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:09.210283 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:09.210465 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:09.211099 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:13:09.211121 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:13:09.212240 (+ 1119us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:13:09.215708 (+ 3468us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:13:09.215715 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:13:09.215723 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:13:09.216280 (+ 557us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:13:09.216706 (+ 426us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:13:09.216711 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:13:09.216718 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:13:09.217314 (+ 596us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:13:09.218516 (+ 1202us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:13:09.218520 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:13:09.218670 (+ 150us) client_negotiation.cc:769] Sending connection context
0807 18:13:09.220017 (+ 1347us) client_negotiation.cc:240] Negotiation successful
0807 18:13:09.220321 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4742,"thread_start_us":4407,"threads_started":1}
WARNING: no policy specified for oryx/127.18.154.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.154.60@KRBTEST.COM" created.
Aug 07 18:13:09 dist-test-slave-gkw7 krb5kdc[27700](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126386, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.154.62@KRBTEST.COM for oryx/127.18.154.61@KRBTEST.COM
I20260807 18:13:09.244490 28507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:09.225962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52220 (local address 127.18.154.61:38131)
0807 18:13:09.226547 (+ 585us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:09.226551 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:09.226569 (+ 18us) server_negotiation.cc:407] Connection header received
0807 18:13:09.226887 (+ 318us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:09.226893 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:09.227022 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:09.227109 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:09.228757 (+ 1648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:09.230070 (+ 1313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:09.232280 (+ 2210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:09.233096 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:09.233257 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:09.238043 (+ 4786us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:13:09.238048 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:13:09.238052 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:13:09.238087 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:13:09.241201 (+ 3114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:09.242131 (+ 930us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:09.242149 (+ 18us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:09.242154 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:09.242379 (+ 225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:09.242822 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:09.242829 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:09.242833 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:09.243142 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:13:09.243308 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0807 18:13:09.243833 (+ 525us) server_negotiation.cc:299] Negotiation successful
0807 18:13:09.244080 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":487}
I20260807 18:13:09.244550 28534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:09.225454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.61:38131 (local address 127.0.0.1:52220)
0807 18:13:09.226288 (+ 834us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:09.226354 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0807 18:13:09.226551 (+ 197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:13:09.227223 (+ 672us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:13:09.227233 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:13:09.228239 (+ 1006us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:13:09.228574 (+ 335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:09.228590 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:09.230229 (+ 1639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:09.230236 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:09.232076 (+ 1840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:09.232094 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:09.233329 (+ 1235us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:09.233338 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:09.233616 (+ 278us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:09.234476 (+ 860us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:13:09.234513 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:13:09.237820 (+ 3307us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:13:09.241362 (+ 3542us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:13:09.241380 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:13:09.241389 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:13:09.242011 (+ 622us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:13:09.242506 (+ 495us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:13:09.242511 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:13:09.242515 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:13:09.242681 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:13:09.243296 (+ 615us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:13:09.243306 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:13:09.243508 (+ 202us) client_negotiation.cc:769] Sending connection context
0807 18:13:09.243699 (+ 191us) client_negotiation.cc:240] Negotiation successful
0807 18:13:09.244083 (+ 384us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":516,"thread_start_us":332,"threads_started":1}
I20260807 18:13:09.246867 27766 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.18.154.61:38131: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "a6983037814748628555659bcf04b165"
member_type: NON_VOTER
last_known_addr {
host: "127.18.154.61"
port: 38131
}
attrs {
promote: true
}
}
dest_uuid: "cc97b2cc29884038b29e0e18de9a6d18"
cas_config_opid_index: -1
I20260807 18:13:09.248277 27766 consensus_queue.cc:237] T 00000000000000000000000000000000 P cc97b2cc29884038b29e0e18de9a6d18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "cc97b2cc29884038b29e0e18de9a6d18" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 34661 } } peers { permanent_uuid: "a6983037814748628555659bcf04b165" member_type: NON_VOTER last_known_addr { host: "127.18.154.61" port: 38131 } attrs { promote: true } }
I20260807 18:13:09.257560 28472 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a6983037814748628555659bcf04b165 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:13:09.259337 28538 raft_consensus.cc:2932] T 00000000000000000000000000000000 P cc97b2cc29884038b29e0e18de9a6d18 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER a6983037814748628555659bcf04b165 (127.18.154.61) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "cc97b2cc29884038b29e0e18de9a6d18" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 34661 } } peers { permanent_uuid: "a6983037814748628555659bcf04b165" member_type: NON_VOTER last_known_addr { host: "127.18.154.61" port: 38131 } attrs { promote: true } } }
I20260807 18:13:09.262089 28472 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a6983037814748628555659bcf04b165 [term 1 FOLLOWER]: Refusing update from remote peer cc97b2cc29884038b29e0e18de9a6d18: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260807 18:13:09.263370 28537 consensus_queue.cc:1035] T 00000000000000000000000000000000 P cc97b2cc29884038b29e0e18de9a6d18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6983037814748628555659bcf04b165" member_type: NON_VOTER last_known_addr { host: "127.18.154.61" port: 38131 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:13:09.265830 28538 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.18.154.61:38131
I20260807 18:13:09.268544 28411 master.cc:537] Master@127.18.154.61:38131 shutting down...
I20260807 18:13:09.270737 28540 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc97b2cc29884038b29e0e18de9a6d18 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "cc97b2cc29884038b29e0e18de9a6d18" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "cc97b2cc29884038b29e0e18de9a6d18" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 34661 } } peers { permanent_uuid: "a6983037814748628555659bcf04b165" member_type: NON_VOTER last_known_addr { host: "127.18.154.61" port: 38131 } attrs { promote: true } } }
I20260807 18:13:09.271569 28540 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc97b2cc29884038b29e0e18de9a6d18 [sys.catalog]: This master's current role is: LEADER
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.154.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/krb5kdc/oryx_127.18.154.60.keytab.
Entry for principal oryx/127.18.154.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/krb5kdc/oryx_127.18.154.60.keytab.
I20260807 18:13:09.280025 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.154.60:36639
--webserver_interface=127.18.154.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/krb5kdc/oryx_127.18.154.60.keytab
--principal=oryx/127.18.154.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.154.20:38893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.154.62:34661,127.18.154.61:38131,127.18.154.60:36639
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 18:13:09.321249 28472 raft_consensus.cc:2932] T 00000000000000000000000000000000 P a6983037814748628555659bcf04b165 [term 1 LEARNER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER a6983037814748628555659bcf04b165 (127.18.154.61) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "cc97b2cc29884038b29e0e18de9a6d18" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 34661 } } peers { permanent_uuid: "a6983037814748628555659bcf04b165" member_type: NON_VOTER last_known_addr { host: "127.18.154.61" port: 38131 } attrs { promote: true } } }
I20260807 18:13:09.343256 28511 sys_catalog.cc:455] T 00000000000000000000000000000000 P a6983037814748628555659bcf04b165 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cc97b2cc29884038b29e0e18de9a6d18. Latest consensus state: current_term: 1 leader_uuid: "cc97b2cc29884038b29e0e18de9a6d18" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "cc97b2cc29884038b29e0e18de9a6d18" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 34661 } } peers { permanent_uuid: "a6983037814748628555659bcf04b165" member_type: NON_VOTER last_known_addr { host: "127.18.154.61" port: 38131 } attrs { promote: true } } }
I20260807 18:13:09.343364 28546 sys_catalog.cc:455] T 00000000000000000000000000000000 P a6983037814748628555659bcf04b165 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "cc97b2cc29884038b29e0e18de9a6d18" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "cc97b2cc29884038b29e0e18de9a6d18" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 34661 } } peers { permanent_uuid: "a6983037814748628555659bcf04b165" member_type: NON_VOTER last_known_addr { host: "127.18.154.61" port: 38131 } attrs { promote: true } } }
I20260807 18:13:09.343607 28511 sys_catalog.cc:458] T 00000000000000000000000000000000 P a6983037814748628555659bcf04b165 [sys.catalog]: This master's current role is: LEARNER
I20260807 18:13:09.343686 28546 sys_catalog.cc:458] T 00000000000000000000000000000000 P a6983037814748628555659bcf04b165 [sys.catalog]: This master's current role is: LEARNER
I20260807 18:13:09.343842 28511 sys_catalog.cc:455] T 00000000000000000000000000000000 P a6983037814748628555659bcf04b165 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "cc97b2cc29884038b29e0e18de9a6d18" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "cc97b2cc29884038b29e0e18de9a6d18" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 34661 } } peers { permanent_uuid: "a6983037814748628555659bcf04b165" member_type: NON_VOTER last_known_addr { host: "127.18.154.61" port: 38131 } attrs { promote: true } } }
I20260807 18:13:09.344034 28511 sys_catalog.cc:458] T 00000000000000000000000000000000 P a6983037814748628555659bcf04b165 [sys.catalog]: This master's current role is: LEARNER
I20260807 18:13:09.347084 28411 raft_consensus.cc:2229] T 00000000000000000000000000000000 P a6983037814748628555659bcf04b165 [term 1 LEARNER]: Raft consensus shutting down.
I20260807 18:13:09.347558 28411 raft_consensus.cc:2258] T 00000000000000000000000000000000 P a6983037814748628555659bcf04b165 [term 1 LEARNER]: Raft consensus is shut down!
I20260807 18:13:09.347752 28411 tablet_replica.cc:330] T 00000000000000000000000000000000 P a6983037814748628555659bcf04b165: stopping tablet replica
I20260807 18:13:09.355485 28411 master.cc:552] Master@127.18.154.61:38131 shutdown complete.
I20260807 18:13:09.355652 28411 master_runner.cc:305] Clearing existing system tablet
I20260807 18:13:09.356194 28411 fs_manager.cc:359] Metadata directory not provided
I20260807 18:13:09.356323 28411 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:13:09.358812 28411 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260807 18:13:09.360380 28553 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:13:09.360692 28411 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:13:09.361013 28411 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.002s sys 0.000s
I20260807 18:13:09.361104 28411 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/master-1/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/master-1/wal
uuid: "a6983037814748628555659bcf04b165"
format_stamp: "Formatted at 2026-08-07 18:13:09 on dist-test-slave-gkw7"
server_key: "a88e101c16b9b6cf938cbe366e1885c9"
server_key_iv: "f57e24ed41fe939be511907bf4819990"
server_key_version: "encryptionkey@0"
I20260807 18:13:09.364770 28411 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P a6983037814748628555659bcf04b165: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:13:09.370041 28411 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P a6983037814748628555659bcf04b165: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260807 18:13:09.370272 28411 log.cc:1196] T 00000000000000000000000000000000 P a6983037814748628555659bcf04b165: Deleting WAL directory at /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260807 18:13:09.370820 28411 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P a6983037814748628555659bcf04b165: Deleting consensus metadata
I20260807 18:13:09.371086 28411 master_runner.cc:315] Copying system tablet from 127.18.154.62:34661
I20260807 18:13:09.378057 28411 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P a6983037814748628555659bcf04b165: tablet copy: Beginning tablet copy session from remote peer at address 127.18.154.62:34661
I20260807 18:13:09.399542 28508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:09.379194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52304 (local address 127.18.154.62:34661)
0807 18:13:09.379447 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:09.379456 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:09.387336 (+ 7880us) server_negotiation.cc:407] Connection header received
0807 18:13:09.387585 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:09.387591 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:09.387682 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:09.387797 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:09.388731 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:09.389477 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:09.390855 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:09.391288 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:09.391426 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:09.393758 (+ 2332us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:13:09.393769 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:13:09.393772 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:13:09.393806 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:13:09.396525 (+ 2719us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:09.397272 (+ 747us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:09.397277 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:09.397281 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:09.397471 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:09.397962 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:09.397967 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:09.397970 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:09.398316 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:13:09.398480 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0807 18:13:09.398903 (+ 423us) server_negotiation.cc:299] Negotiation successful
0807 18:13:09.399141 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 18:13:09.399906 28560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:09.379145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:34661 (local address 127.0.0.1:52304)
0807 18:13:09.387186 (+ 8041us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:09.387209 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 18:13:09.387415 (+ 206us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:13:09.387833 (+ 418us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:13:09.387837 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:13:09.388334 (+ 497us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:13:09.388561 (+ 227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:09.388571 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:09.389686 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:09.389692 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:09.390699 (+ 1007us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:09.390711 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:09.391452 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:09.391458 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:09.391649 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:09.392515 (+ 866us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:13:09.392534 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:13:09.393521 (+ 987us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:13:09.396667 (+ 3146us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:13:09.396672 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:13:09.396676 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:13:09.397145 (+ 469us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:13:09.397667 (+ 522us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:13:09.397672 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:13:09.397679 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:13:09.397836 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:13:09.398448 (+ 612us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:13:09.398453 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:13:09.398619 (+ 166us) client_negotiation.cc:769] Sending connection context
0807 18:13:09.398805 (+ 186us) client_negotiation.cc:240] Negotiation successful
0807 18:13:09.399141 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7883,"thread_start_us":7732,"threads_started":1}
I20260807 18:13:09.400882 27790 tablet_copy_service.cc:140] P cc97b2cc29884038b29e0e18de9a6d18: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer a6983037814748628555659bcf04b165 ({username='oryx', principal='oryx/127.18.154.61@KRBTEST.COM'} at 127.0.0.1:52304)
I20260807 18:13:09.401294 27790 tablet_copy_service.cc:161] P cc97b2cc29884038b29e0e18de9a6d18: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer a6983037814748628555659bcf04b165 at {username='oryx', principal='oryx/127.18.154.61@KRBTEST.COM'} at 127.0.0.1:52304: session id = a6983037814748628555659bcf04b165-00000000000000000000000000000000
I20260807 18:13:09.405124 27790 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P cc97b2cc29884038b29e0e18de9a6d18: Tablet Copy: opened 0 blocks and 1 log segments
I20260807 18:13:09.407685 28411 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260807 18:13:09.414057 28411 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P a6983037814748628555659bcf04b165: tablet copy: Starting download of 0 data blocks...
I20260807 18:13:09.414559 28411 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P a6983037814748628555659bcf04b165: tablet copy: Starting download of 1 WAL segments...
I20260807 18:13:09.418238 28411 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P a6983037814748628555659bcf04b165: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260807 18:13:09.422331 27790 tablet_copy_service.cc:342] P cc97b2cc29884038b29e0e18de9a6d18: Request end of tablet copy session a6983037814748628555659bcf04b165-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.18.154.61@KRBTEST.COM'} at 127.0.0.1:52304
I20260807 18:13:09.422600 27790 tablet_copy_service.cc:433] P cc97b2cc29884038b29e0e18de9a6d18: ending tablet copy session a6983037814748628555659bcf04b165-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer a6983037814748628555659bcf04b165
I20260807 18:13:09.433688 28411 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:13:09.437492 28562 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:13:09.438505 28563 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:13:09.438946 28565 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:13:09.439970 28411 server_base.cc:1034] running on GCE node
I20260807 18:13:09.440308 28411 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:13:09.440935 28411 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:13:09.442165 28411 hybrid_clock.cc:648] HybridClock initialized: now 1786126389442100 us; error 35 us; skew 500 ppm
I20260807 18:13:09.442442 28411 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:13:09 dist-test-slave-gkw7 krb5kdc[27700](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126389, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.154.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:13:09.444447 28411 init.cc:377] Logged in from keytab as oryx/127.18.154.61@KRBTEST.COM (short username oryx)
I20260807 18:13:09.446444 28411 webserver.cc:460] Webserver started at http://127.18.154.61:45961/ using document root <none> and password file <none>
I20260807 18:13:09.446779 28411 fs_manager.cc:359] Metadata directory not provided
I20260807 18:13:09.446875 28411 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:13:09.449690 28411 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.002s
I20260807 18:13:09.451385 28575 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:13:09.451812 28411 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.002s sys 0.000s
I20260807 18:13:09.451960 28411 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/master-1/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/master-1/wal
uuid: "a6983037814748628555659bcf04b165"
format_stamp: "Formatted at 2026-08-07 18:13:09 on dist-test-slave-gkw7"
server_key: "a88e101c16b9b6cf938cbe366e1885c9"
server_key_iv: "f57e24ed41fe939be511907bf4819990"
server_key_version: "encryptionkey@0"
I20260807 18:13:09.452142 28411 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:13:09.471796 28411 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:13:09.475611 28411 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:13:09.506956 28411 rpc_server.cc:307] RPC server started. Bound to: 127.18.154.61:38131
I20260807 18:13:09.506983 28644 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.154.61:38131 every 8 connection(s)
I20260807 18:13:09.508116 28411 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/master-1/data/info.pb
I20260807 18:13:09.516503 28645 sys_catalog.cc:263] Verifying existing consensus state
I20260807 18:13:09.522115 28645 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a6983037814748628555659bcf04b165: Bootstrap starting.
I20260807 18:13:09.536617 28645 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a6983037814748628555659bcf04b165: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 18:13:09.537457 28645 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a6983037814748628555659bcf04b165: Bootstrap complete.
I20260807 18:13:09.538790 28645 raft_consensus.cc:348] T 00000000000000000000000000000000 P a6983037814748628555659bcf04b165 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "cc97b2cc29884038b29e0e18de9a6d18" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 34661 } } peers { permanent_uuid: "a6983037814748628555659bcf04b165" member_type: NON_VOTER last_known_addr { host: "127.18.154.61" port: 38131 } attrs { promote: true } }
I20260807 18:13:09.539155 28645 raft_consensus.cc:729] T 00000000000000000000000000000000 P a6983037814748628555659bcf04b165 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: a6983037814748628555659bcf04b165, State: Initialized, Role: LEARNER
I20260807 18:13:09.539515 28645 consensus_queue.cc:260] T 00000000000000000000000000000000 P a6983037814748628555659bcf04b165 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "cc97b2cc29884038b29e0e18de9a6d18" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 34661 } } peers { permanent_uuid: "a6983037814748628555659bcf04b165" member_type: NON_VOTER last_known_addr { host: "127.18.154.61" port: 38131 } attrs { promote: true } }
I20260807 18:13:09.547067 28648 sys_catalog.cc:455] T 00000000000000000000000000000000 P a6983037814748628555659bcf04b165 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "cc97b2cc29884038b29e0e18de9a6d18" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 34661 } } peers { permanent_uuid: "a6983037814748628555659bcf04b165" member_type: NON_VOTER last_known_addr { host: "127.18.154.61" port: 38131 } attrs { promote: true } } }
I20260807 18:13:09.547487 28645 sys_catalog.cc:564] T 00000000000000000000000000000000 P a6983037814748628555659bcf04b165 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:13:09.547633 28648 sys_catalog.cc:458] T 00000000000000000000000000000000 P a6983037814748628555659bcf04b165 [sys.catalog]: This master's current role is: LEARNER
I20260807 18:13:09.568385 28405 heartbeater.cc:503] Master 127.18.154.62:34661 was elected leader, sending a full tablet report...
I20260807 18:13:09.569773 28661 catalog_manager.cc:1259] Loaded cluster ID: 6f80460f0d5e43d69126321a9a079ffa
I20260807 18:13:09.569974 28661 catalog_manager.cc:1552] T 00000000000000000000000000000000 P a6983037814748628555659bcf04b165: loading cluster ID for follower catalog manager: success
I20260807 18:13:09.574211 28661 catalog_manager.cc:1574] T 00000000000000000000000000000000 P a6983037814748628555659bcf04b165: acquiring CA information for follower catalog manager: success
I20260807 18:13:09.577291 28661 catalog_manager.cc:1602] T 00000000000000000000000000000000 P a6983037814748628555659bcf04b165: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260807 18:13:09.756332 28544 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:13:09.756677 28544 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:13:09.756829 28544 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:13:09.764559 28544 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:13:09.764721 28544 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:13:09.764820 28544 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:13:09.764920 28544 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:13:09.765048 28544 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:13:09.776685 28544 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.154.20:38893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.18.154.62:34661,127.18.154.61:38131,127.18.154.60:36639
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/krb5kdc/oryx_127.18.154.60.keytab
--principal=oryx/127.18.154.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.154.60:36639
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.18.154.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:13:09.778246 28544 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:13:09.779721 28544 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:13:09.794524 28669 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:13:09.794675 28668 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:13:09.795086 28671 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:13:09.795884 28544 server_base.cc:1034] running on GCE node
I20260807 18:13:09.796772 28544 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:13:09.798308 28544 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:13:09.799530 28544 hybrid_clock.cc:648] HybridClock initialized: now 1786126389799474 us; error 39 us; skew 500 ppm
I20260807 18:13:09.799996 28544 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:13:09 dist-test-slave-gkw7 krb5kdc[27700](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126389, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.154.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:13:09.803835 28544 init.cc:377] Logged in from keytab as oryx/127.18.154.60@KRBTEST.COM (short username oryx)
I20260807 18:13:09.806051 28544 webserver.cc:460] Webserver started at http://127.18.154.60:36961/ using document root <none> and password file <none>
I20260807 18:13:09.806823 28544 fs_manager.cc:359] Metadata directory not provided
I20260807 18:13:09.806960 28544 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:13:09.807384 28544 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:13:09.810906 28544 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/master-2/data/instance:
uuid: "3d6d83d5ddcb4df3bcba443888204a34"
format_stamp: "Formatted at 2026-08-07 18:13:09 on dist-test-slave-gkw7"
server_key: "621de20b74cb30499eabcb746c2cbeed"
server_key_iv: "27bd2600f132223f7b6f683df9faee12"
server_key_version: "encryptionkey@0"
I20260807 18:13:09.811823 28544 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/master-2/wal/instance:
uuid: "3d6d83d5ddcb4df3bcba443888204a34"
format_stamp: "Formatted at 2026-08-07 18:13:09 on dist-test-slave-gkw7"
server_key: "621de20b74cb30499eabcb746c2cbeed"
server_key_iv: "27bd2600f132223f7b6f683df9faee12"
server_key_version: "encryptionkey@0"
I20260807 18:13:09.818060 28544 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.005s
I20260807 18:13:09.822880 28682 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:13:09.824522 28544 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.003s
I20260807 18:13:09.824743 28544 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/master-2/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/master-2/wal
uuid: "3d6d83d5ddcb4df3bcba443888204a34"
format_stamp: "Formatted at 2026-08-07 18:13:09 on dist-test-slave-gkw7"
server_key: "621de20b74cb30499eabcb746c2cbeed"
server_key_iv: "27bd2600f132223f7b6f683df9faee12"
server_key_version: "encryptionkey@0"
I20260807 18:13:09.824967 28544 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:13:09.851864 28544 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:13:09.857682 28544 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:13:09.857999 28544 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:13:09.882761 28544 rpc_server.cc:307] RPC server started. Bound to: 127.18.154.60:36639
I20260807 18:13:09.882710 28758 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.154.60:36639 every 8 connection(s)
I20260807 18:13:09.885015 28544 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/master-2/data/info.pb
I20260807 18:13:09.886973 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 28544
I20260807 18:13:09.889688 28748 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc97b2cc29884038b29e0e18de9a6d18 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "cc97b2cc29884038b29e0e18de9a6d18" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "cc97b2cc29884038b29e0e18de9a6d18" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 34661 } } peers { permanent_uuid: "a6983037814748628555659bcf04b165" member_type: NON_VOTER last_known_addr { host: "127.18.154.61" port: 38131 } attrs { promote: true } } }
I20260807 18:13:09.890882 28748 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc97b2cc29884038b29e0e18de9a6d18 [sys.catalog]: This master's current role is: LEADER
I20260807 18:13:09.903241 28764 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:13:09.910804 28762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:09.883682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.61:38131 (local address 127.0.0.1:52230)
0807 18:13:09.898044 (+ 14362us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:09.898072 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0807 18:13:09.898260 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:13:09.900205 (+ 1945us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:13:09.900213 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:13:09.900264 (+ 51us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:13:09.900562 (+ 298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:09.900577 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:09.903495 (+ 2918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:09.903504 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:09.906470 (+ 2966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:09.906497 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:09.909425 (+ 2928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:09.909436 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:09.909791 (+ 355us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:09.909894 (+ 103us) client_negotiation.cc:769] Sending connection context
0807 18:13:09.910020 (+ 126us) client_negotiation.cc:240] Negotiation successful
0807 18:13:09.910261 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14219,"thread_start_us":14685,"threads_started":1}
I20260807 18:13:09.911995 28763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:09.883822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52230 (local address 127.18.154.61:38131)
0807 18:13:09.899415 (+ 15593us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:09.899423 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:09.899445 (+ 22us) server_negotiation.cc:407] Connection header received
0807 18:13:09.899522 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:09.899527 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:09.899636 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:09.899779 (+ 143us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:13:09.900817 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:09.903199 (+ 2382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:09.906749 (+ 3550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:09.908691 (+ 1942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:09.908885 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:09.909268 (+ 383us) server_negotiation.cc:1035] Waiting for connection context
0807 18:13:09.911179 (+ 1911us) server_negotiation.cc:299] Negotiation successful
0807 18:13:09.911489 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15443,"thread_start_us":15250,"threads_started":1}
I20260807 18:13:09.913419 28648 sys_catalog.cc:455] T 00000000000000000000000000000000 P a6983037814748628555659bcf04b165 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cc97b2cc29884038b29e0e18de9a6d18. Latest consensus state: current_term: 1 leader_uuid: "cc97b2cc29884038b29e0e18de9a6d18" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "cc97b2cc29884038b29e0e18de9a6d18" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 34661 } } peers { permanent_uuid: "a6983037814748628555659bcf04b165" member_type: NON_VOTER last_known_addr { host: "127.18.154.61" port: 38131 } attrs { promote: true } } }
I20260807 18:13:09.913780 28648 sys_catalog.cc:458] T 00000000000000000000000000000000 P a6983037814748628555659bcf04b165 [sys.catalog]: This master's current role is: LEARNER
I20260807 18:13:09.914857 28752 raft_consensus.cc:1050] T 00000000000000000000000000000000 P cc97b2cc29884038b29e0e18de9a6d18: attempting to promote NON_VOTER a6983037814748628555659bcf04b165 to VOTER
Aug 07 18:13:09 dist-test-slave-gkw7 krb5kdc[27700](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126386, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.18.154.60@KRBTEST.COM
I20260807 18:13:09.915926 28752 consensus_queue.cc:237] T 00000000000000000000000000000000 P cc97b2cc29884038b29e0e18de9a6d18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "cc97b2cc29884038b29e0e18de9a6d18" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 34661 } } peers { permanent_uuid: "a6983037814748628555659bcf04b165" member_type: VOTER last_known_addr { host: "127.18.154.61" port: 38131 } attrs { promote: false } }
I20260807 18:13:09.916008 28764 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.154.62" port: 34661 } has no permanent_uuid. Determining permanent_uuid...
I20260807 18:13:09.918308 28611 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a6983037814748628555659bcf04b165 [term 1 LEARNER]: Refusing update from remote peer cc97b2cc29884038b29e0e18de9a6d18: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260807 18:13:09.919013 28759 consensus_queue.cc:1035] T 00000000000000000000000000000000 P cc97b2cc29884038b29e0e18de9a6d18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6983037814748628555659bcf04b165" member_type: VOTER last_known_addr { host: "127.18.154.61" port: 38131 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260807 18:13:09.925137 28752 raft_consensus.cc:2932] T 00000000000000000000000000000000 P cc97b2cc29884038b29e0e18de9a6d18 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, a6983037814748628555659bcf04b165 (127.18.154.61) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "cc97b2cc29884038b29e0e18de9a6d18" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 34661 } } peers { permanent_uuid: "a6983037814748628555659bcf04b165" member_type: VOTER last_known_addr { host: "127.18.154.61" port: 38131 } attrs { promote: false } } }
I20260807 18:13:09.925933 28611 raft_consensus.cc:2932] T 00000000000000000000000000000000 P a6983037814748628555659bcf04b165 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, a6983037814748628555659bcf04b165 (127.18.154.61) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "cc97b2cc29884038b29e0e18de9a6d18" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 34661 } } peers { permanent_uuid: "a6983037814748628555659bcf04b165" member_type: VOTER last_known_addr { host: "127.18.154.61" port: 38131 } attrs { promote: false } } }
I20260807 18:13:09.929445 28752 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc97b2cc29884038b29e0e18de9a6d18 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "cc97b2cc29884038b29e0e18de9a6d18" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "cc97b2cc29884038b29e0e18de9a6d18" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 34661 } } peers { permanent_uuid: "a6983037814748628555659bcf04b165" member_type: VOTER last_known_addr { host: "127.18.154.61" port: 38131 } attrs { promote: false } } }
I20260807 18:13:09.929757 28752 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc97b2cc29884038b29e0e18de9a6d18 [sys.catalog]: This master's current role is: LEADER
I20260807 18:13:09.929656 28748 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc97b2cc29884038b29e0e18de9a6d18 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "cc97b2cc29884038b29e0e18de9a6d18" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "cc97b2cc29884038b29e0e18de9a6d18" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 34661 } } peers { permanent_uuid: "a6983037814748628555659bcf04b165" member_type: VOTER last_known_addr { host: "127.18.154.61" port: 38131 } attrs { promote: false } } }
I20260807 18:13:09.929868 28748 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc97b2cc29884038b29e0e18de9a6d18 [sys.catalog]: This master's current role is: LEADER
I20260807 18:13:09.930629 28770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:09.895316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32846 (local address 127.18.154.60:36639)
0807 18:13:09.898093 (+ 2777us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:09.898117 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:09.898181 (+ 64us) server_negotiation.cc:407] Connection header received
0807 18:13:09.901117 (+ 2936us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:09.901153 (+ 36us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:09.901798 (+ 645us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:09.904689 (+ 2891us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:09.905282 (+ 593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:09.907621 (+ 2339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:09.909694 (+ 2073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:09.910930 (+ 1236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:09.915582 (+ 4652us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:09.917806 (+ 2224us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:13:09.917821 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:13:09.917846 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:13:09.917903 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:13:09.923479 (+ 5576us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:09.924502 (+ 1023us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:09.924511 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:09.924520 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:09.924765 (+ 245us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:09.925393 (+ 628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:09.925398 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:09.925400 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:09.926474 (+ 1074us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:13:09.927895 (+ 1421us) server_negotiation.cc:1035] Waiting for connection context
0807 18:13:09.928219 (+ 324us) server_negotiation.cc:299] Negotiation successful
0807 18:13:09.928733 (+ 514us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2474,"thread_start_us":2124,"threads_started":1}
I20260807 18:13:09.930979 28648 sys_catalog.cc:455] T 00000000000000000000000000000000 P a6983037814748628555659bcf04b165 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "cc97b2cc29884038b29e0e18de9a6d18" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "cc97b2cc29884038b29e0e18de9a6d18" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 34661 } } peers { permanent_uuid: "a6983037814748628555659bcf04b165" member_type: VOTER last_known_addr { host: "127.18.154.61" port: 38131 } attrs { promote: false } } }
I20260807 18:13:09.931231 28648 sys_catalog.cc:458] T 00000000000000000000000000000000 P a6983037814748628555659bcf04b165 [sys.catalog]: This master's current role is: FOLLOWER
Aug 07 18:13:09 dist-test-slave-gkw7 krb5kdc[27700](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126389, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.154.60@KRBTEST.COM for oryx/127.18.154.62@KRBTEST.COM
I20260807 18:13:09.941262 28772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:09.920264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:34661 (local address 127.0.0.1:52320)
0807 18:13:09.922343 (+ 2079us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:09.922399 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0807 18:13:09.922569 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:13:09.923979 (+ 1410us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:13:09.923986 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:13:09.924970 (+ 984us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:13:09.925590 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:09.925605 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:09.927450 (+ 1845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:09.927456 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:09.929451 (+ 1995us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:09.929468 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:09.931024 (+ 1556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:09.931031 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:09.932309 (+ 1278us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:09.933094 (+ 785us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:13:09.933120 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:13:09.935797 (+ 2677us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:13:09.938862 (+ 3065us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:13:09.938872 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:13:09.938878 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:13:09.939359 (+ 481us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:13:09.939697 (+ 338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:13:09.939700 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:13:09.939702 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:13:09.939825 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:13:09.940367 (+ 542us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:13:09.940373 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:13:09.940530 (+ 157us) client_negotiation.cc:769] Sending connection context
0807 18:13:09.940687 (+ 157us) client_negotiation.cc:240] Negotiation successful
0807 18:13:09.940936 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1749,"thread_start_us":892,"threads_started":1}
I20260807 18:13:09.941270 28771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:09.920272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52320 (local address 127.18.154.62:34661)
0807 18:13:09.920873 (+ 601us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:09.920879 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:09.923641 (+ 2762us) server_negotiation.cc:407] Connection header received
0807 18:13:09.923703 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:09.923708 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:09.923796 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:09.923918 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:09.925828 (+ 1910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:09.927277 (+ 1449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:09.929734 (+ 2457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:09.930593 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:09.930734 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:09.935995 (+ 5261us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:13:09.936001 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:13:09.936004 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:13:09.936037 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:13:09.938742 (+ 2705us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:09.939468 (+ 726us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:09.939472 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:09.939474 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:09.939604 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:09.939955 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:09.939959 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:09.939961 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:09.940258 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:13:09.940384 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0807 18:13:09.940790 (+ 406us) server_negotiation.cc:299] Negotiation successful
0807 18:13:09.940957 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":466,"thread_start_us":278,"threads_started":1}
I20260807 18:13:09.942801 28764 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.154.61" port: 38131 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 18:13:09 dist-test-slave-gkw7 krb5kdc[27700](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126389, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.154.60@KRBTEST.COM for oryx/127.18.154.61@KRBTEST.COM
I20260807 18:13:09.957484 28763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:09.943710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52246 (local address 127.18.154.61:38131)
0807 18:13:09.943886 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:09.943890 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:09.943906 (+ 16us) server_negotiation.cc:407] Connection header received
0807 18:13:09.943954 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:09.943958 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:09.944015 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:09.944092 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:09.944862 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:09.945742 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:09.947167 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:09.947671 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:09.947816 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:09.951338 (+ 3522us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:13:09.951343 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:13:09.951346 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:13:09.951381 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:13:09.954574 (+ 3193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:09.955394 (+ 820us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:09.955401 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:09.955405 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:09.955581 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:09.955980 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:09.955985 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:09.955988 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:09.956284 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:13:09.956443 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0807 18:13:09.956827 (+ 384us) server_negotiation.cc:299] Negotiation successful
0807 18:13:09.957082 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 18:13:09.957475 28772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:09.943440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.61:38131 (local address 127.0.0.1:52246)
0807 18:13:09.943631 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:09.943647 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 18:13:09.943733 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:13:09.944173 (+ 440us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:13:09.944176 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:13:09.944508 (+ 332us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:13:09.944666 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:09.944674 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:09.945870 (+ 1196us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:09.945874 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:09.946980 (+ 1106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:09.946991 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:09.947860 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:09.947864 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:09.948019 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:09.948700 (+ 681us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:13:09.948717 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:13:09.951115 (+ 2398us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:13:09.954720 (+ 3605us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:13:09.954728 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:13:09.954733 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:13:09.955265 (+ 532us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:13:09.955702 (+ 437us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:13:09.955708 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:13:09.955711 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:13:09.955861 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:13:09.956420 (+ 559us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:13:09.956425 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:13:09.956576 (+ 151us) client_negotiation.cc:769] Sending connection context
0807 18:13:09.956725 (+ 149us) client_negotiation.cc:240] Negotiation successful
0807 18:13:09.957101 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60}
I20260807 18:13:09.958696 28764 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.154.60" port: 36639 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 18:13:09 dist-test-slave-gkw7 krb5kdc[27700](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126389, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.154.60@KRBTEST.COM for oryx/127.18.154.60@KRBTEST.COM
I20260807 18:13:09.972683 28772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:09.959476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.60:36639 (local address 127.0.0.1:32854)
0807 18:13:09.959774 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:09.959793 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 18:13:09.959923 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:13:09.960289 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:13:09.960293 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:13:09.960733 (+ 440us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:13:09.960879 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:09.960887 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:09.961961 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:09.961969 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:09.962905 (+ 936us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:09.962917 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:09.963682 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:09.963687 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:09.963971 (+ 284us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:09.964649 (+ 678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:13:09.964669 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:13:09.967185 (+ 2516us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:13:09.970333 (+ 3148us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:13:09.970337 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:13:09.970340 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:13:09.970782 (+ 442us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:13:09.971117 (+ 335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:13:09.971120 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:13:09.971123 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:13:09.971283 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:13:09.971813 (+ 530us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:13:09.971839 (+ 26us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:13:09.971978 (+ 139us) client_negotiation.cc:769] Sending connection context
0807 18:13:09.972099 (+ 121us) client_negotiation.cc:240] Negotiation successful
0807 18:13:09.972351 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":71}
I20260807 18:13:09.973104 28770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:09.959810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32854 (local address 127.18.154.60:36639)
0807 18:13:09.960002 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:09.960010 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:09.960026 (+ 16us) server_negotiation.cc:407] Connection header received
0807 18:13:09.960113 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:09.960116 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:09.960167 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:09.960270 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:09.961012 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:09.961843 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:09.963040 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:09.963505 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:09.963651 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:09.967357 (+ 3706us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:13:09.967362 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:13:09.967365 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:13:09.967394 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:13:09.970188 (+ 2794us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:09.970887 (+ 699us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:09.970891 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:09.970893 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:09.971018 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:09.971375 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:09.971378 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:09.971380 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:09.971655 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:13:09.971802 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0807 18:13:09.972183 (+ 381us) server_negotiation.cc:299] Negotiation successful
0807 18:13:09.972502 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":109,"server-negotiator.queue_time_us":68}
I20260807 18:13:09.978653 28764 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d6d83d5ddcb4df3bcba443888204a34: Bootstrap starting.
I20260807 18:13:09.982161 28764 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3d6d83d5ddcb4df3bcba443888204a34: Neither blocks nor log segments found. Creating new log.
I20260807 18:13:09.983327 28764 log.cc:824] T 00000000000000000000000000000000 P 3d6d83d5ddcb4df3bcba443888204a34: Log is configured to *not* fsync() on all Append() calls
I20260807 18:13:09.986474 28764 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d6d83d5ddcb4df3bcba443888204a34: No bootstrap required, opened a new log
I20260807 18:13:09.994496 28764 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3d6d83d5ddcb4df3bcba443888204a34 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc97b2cc29884038b29e0e18de9a6d18" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 34661 } } peers { permanent_uuid: "a6983037814748628555659bcf04b165" member_type: VOTER last_known_addr { host: "127.18.154.61" port: 38131 } } peers { permanent_uuid: "3d6d83d5ddcb4df3bcba443888204a34" member_type: VOTER last_known_addr { host: "127.18.154.60" port: 36639 } }
I20260807 18:13:09.994912 28764 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3d6d83d5ddcb4df3bcba443888204a34 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:13:09.995082 28764 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3d6d83d5ddcb4df3bcba443888204a34 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d6d83d5ddcb4df3bcba443888204a34, State: Initialized, Role: FOLLOWER
I20260807 18:13:09.995718 28764 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3d6d83d5ddcb4df3bcba443888204a34 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc97b2cc29884038b29e0e18de9a6d18" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 34661 } } peers { permanent_uuid: "a6983037814748628555659bcf04b165" member_type: VOTER last_known_addr { host: "127.18.154.61" port: 38131 } } peers { permanent_uuid: "3d6d83d5ddcb4df3bcba443888204a34" member_type: VOTER last_known_addr { host: "127.18.154.60" port: 36639 } }
I20260807 18:13:09.996934 28777 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d6d83d5ddcb4df3bcba443888204a34 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc97b2cc29884038b29e0e18de9a6d18" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 34661 } } peers { permanent_uuid: "a6983037814748628555659bcf04b165" member_type: VOTER last_known_addr { host: "127.18.154.61" port: 38131 } } peers { permanent_uuid: "3d6d83d5ddcb4df3bcba443888204a34" member_type: VOTER last_known_addr { host: "127.18.154.60" port: 36639 } } }
I20260807 18:13:09.997394 28777 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d6d83d5ddcb4df3bcba443888204a34 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 18:13:09.998003 28764 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3d6d83d5ddcb4df3bcba443888204a34 [sys.catalog]: configured and running, proceeding with master startup.
W20260807 18:13:10.009895 28796 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3d6d83d5ddcb4df3bcba443888204a34: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 18:13:10.010102 28796 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 18:13:10.013845 28544 master_runner.cc:428] Detected that this master 3d6d83d5ddcb4df3bcba443888204a34 is joining an existing cluster
I20260807 18:13:10.014053 28544 master_runner.cc:432] Initiating AddMaster RPC to add 127.18.154.60:36639
I20260807 18:13:10.037734 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 27856
I20260807 18:13:10.040371 28771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:10.018962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52334 (local address 127.18.154.62:34661)
0807 18:13:10.019151 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:10.019155 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:10.020765 (+ 1610us) server_negotiation.cc:407] Connection header received
0807 18:13:10.021317 (+ 552us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:10.021327 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:10.021452 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:10.021574 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:10.023865 (+ 2291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:10.025177 (+ 1312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:10.027880 (+ 2703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:10.028792 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:10.028917 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:10.031921 (+ 3004us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:13:10.031931 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:13:10.031939 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:13:10.031987 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:13:10.036193 (+ 4206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:10.037129 (+ 936us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:10.037137 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:10.037140 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:10.037304 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:10.037874 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:10.037880 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:10.037883 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:10.038368 (+ 485us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:13:10.038522 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0807 18:13:10.039420 (+ 898us) server_negotiation.cc:299] Negotiation successful
0807 18:13:10.039692 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260807 18:13:10.041668 28804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:10.019456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:34661 (local address 127.0.0.1:52334)
0807 18:13:10.020533 (+ 1077us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:10.020655 (+ 122us) client_negotiation.cc:174] Beginning negotiation
0807 18:13:10.021123 (+ 468us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:13:10.021894 (+ 771us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:13:10.021901 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:13:10.022967 (+ 1066us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:13:10.023632 (+ 665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:10.023651 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:10.025397 (+ 1746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:10.025404 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:10.027653 (+ 2249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:10.027675 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:10.029397 (+ 1722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:10.029407 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:10.029750 (+ 343us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:10.030735 (+ 985us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:13:10.030753 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:13:10.031684 (+ 931us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:13:10.036385 (+ 4701us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:13:10.036394 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:13:10.036398 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:13:10.036979 (+ 581us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:13:10.037460 (+ 481us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:13:10.037466 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:13:10.037471 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:13:10.037731 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:13:10.038626 (+ 895us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:13:10.038633 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:13:10.039053 (+ 420us) client_negotiation.cc:769] Sending connection context
0807 18:13:10.040660 (+ 1607us) client_negotiation.cc:240] Negotiation successful
0807 18:13:10.041035 (+ 375us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":848,"thread_start_us":534,"threads_started":1}
I20260807 18:13:10.054162 28770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:10.020845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32858 (local address 127.18.154.60:36639)
0807 18:13:10.021057 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:10.021065 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:10.032403 (+ 11338us) server_negotiation.cc:407] Connection header received
0807 18:13:10.032607 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:10.032615 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:10.032709 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:10.032787 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:10.034639 (+ 1852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:10.035697 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:10.038160 (+ 2463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:10.038911 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:10.039017 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:10.042628 (+ 3611us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:13:10.042637 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:13:10.042645 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:13:10.042690 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:13:10.049871 (+ 7181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:10.051519 (+ 1648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:10.051524 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:10.051528 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:10.051670 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:10.052055 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:10.052060 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:10.052064 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:10.052556 (+ 492us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:13:10.052750 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0807 18:13:10.053277 (+ 527us) server_negotiation.cc:299] Negotiation successful
0807 18:13:10.053718 (+ 441us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":171,"server-negotiator.queue_time_us":37}
I20260807 18:13:10.054580 28806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:10.020537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.60:36639 (local address 127.0.0.1:32858)
0807 18:13:10.032316 (+ 11779us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:10.032335 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 18:13:10.032474 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:13:10.032998 (+ 524us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:13:10.033002 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:13:10.034088 (+ 1086us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:13:10.034492 (+ 404us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:10.034504 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:10.036275 (+ 1771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:10.036279 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:10.037993 (+ 1714us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:10.038012 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:10.039372 (+ 1360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:10.039380 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:10.039553 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:10.040242 (+ 689us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:13:10.040275 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:13:10.042375 (+ 2100us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:13:10.050089 (+ 7714us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:13:10.050097 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:13:10.050101 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:13:10.051389 (+ 1288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:13:10.051819 (+ 430us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:13:10.051822 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:13:10.051824 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:13:10.051933 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:13:10.052792 (+ 859us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:13:10.052815 (+ 23us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:13:10.053019 (+ 204us) client_negotiation.cc:769] Sending connection context
0807 18:13:10.053173 (+ 154us) client_negotiation.cc:240] Negotiation successful
0807 18:13:10.053519 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11433,"thread_start_us":11556,"threads_started":1}
Aug 07 18:13:10 dist-test-slave-gkw7 krb5kdc[27700](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126386, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.18.154.1@KRBTEST.COM
I20260807 18:13:10.074658 28805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:10.019874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.61:38131 (local address 127.0.0.1:52250)
0807 18:13:10.030491 (+ 10617us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:10.030524 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0807 18:13:10.031665 (+ 1141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:13:10.032283 (+ 618us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:13:10.032294 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:13:10.033287 (+ 993us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:13:10.033913 (+ 626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:10.033926 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:10.035550 (+ 1624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:10.035555 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:10.048661 (+ 13106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:10.048685 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:10.050153 (+ 1468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:10.050161 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:10.050453 (+ 292us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:10.051214 (+ 761us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:13:10.051243 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:13:10.052588 (+ 1345us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:13:10.057256 (+ 4668us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:13:10.057260 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:13:10.057263 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:13:10.071254 (+ 13991us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:13:10.072307 (+ 1053us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:13:10.072314 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:13:10.072319 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:13:10.072529 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:13:10.073330 (+ 801us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:13:10.073336 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:13:10.073577 (+ 241us) client_negotiation.cc:769] Sending connection context
0807 18:13:10.073745 (+ 168us) client_negotiation.cc:240] Negotiation successful
0807 18:13:10.074090 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10406,"thread_start_us":10136,"threads_started":1}
I20260807 18:13:10.076521 28763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:10.019194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52250 (local address 127.18.154.61:38131)
0807 18:13:10.019365 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:10.019371 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:10.030660 (+ 11289us) server_negotiation.cc:407] Connection header received
0807 18:13:10.031877 (+ 1217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:10.031884 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:10.031979 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:10.032054 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:10.034104 (+ 2050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:10.035368 (+ 1264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:10.048849 (+ 13481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:10.049653 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:10.049812 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:10.053069 (+ 3257us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:13:10.053076 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:13:10.053081 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:13:10.053126 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:13:10.057073 (+ 3947us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:10.071877 (+ 14804us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:10.071885 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:10.071890 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:10.072140 (+ 250us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:10.072706 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:10.072711 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:10.072715 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:10.073125 (+ 410us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:13:10.075397 (+ 2272us) server_negotiation.cc:1035] Waiting for connection context
0807 18:13:10.075586 (+ 189us) server_negotiation.cc:299] Negotiation successful
0807 18:13:10.075912 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
Aug 07 18:13:10 dist-test-slave-gkw7 krb5kdc[27700](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126386, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.154.62@KRBTEST.COM for oryx/127.18.154.60@KRBTEST.COM
I20260807 18:13:10.080679 28808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:10.039563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49964 (local address 127.18.154.1:41283)
0807 18:13:10.058116 (+ 18553us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:10.058124 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:10.058150 (+ 26us) server_negotiation.cc:407] Connection header received
0807 18:13:10.058236 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:10.058243 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:10.058467 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:10.058617 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:10.060636 (+ 2019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:10.061926 (+ 1290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:10.064067 (+ 2141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:10.064708 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:10.064816 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:10.070816 (+ 6000us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:13:10.070839 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:13:10.070860 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:13:10.070907 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:13:10.075911 (+ 5004us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:10.077121 (+ 1210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:10.077134 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:10.077138 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:10.077323 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:10.078037 (+ 714us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:10.078044 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:10.078048 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:10.078580 (+ 532us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:13:10.078913 (+ 333us) server_negotiation.cc:1035] Waiting for connection context
0807 18:13:10.079606 (+ 693us) server_negotiation.cc:299] Negotiation successful
0807 18:13:10.079911 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18397,"thread_start_us":18188,"threads_started":1}
I20260807 18:13:10.082554 28021 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:13:10.090870 28762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:10.048692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.60:36639 (local address 127.0.0.1:32868)
0807 18:13:10.049023 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:10.049042 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 18:13:10.049154 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:13:10.050313 (+ 1159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:13:10.050320 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:13:10.050886 (+ 566us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:13:10.051165 (+ 279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:10.051179 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:10.074613 (+ 23434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:10.074624 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:10.076255 (+ 1631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:10.076280 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:10.077632 (+ 1352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:10.077640 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:10.077854 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:10.078613 (+ 759us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:13:10.078643 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:13:10.081412 (+ 2769us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:13:10.087372 (+ 5960us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:13:10.087379 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:13:10.087383 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:13:10.087992 (+ 609us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:13:10.088584 (+ 592us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:13:10.088588 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:13:10.088592 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:13:10.088805 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:13:10.089645 (+ 840us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:13:10.089650 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:13:10.089854 (+ 204us) client_negotiation.cc:769] Sending connection context
0807 18:13:10.090051 (+ 197us) client_negotiation.cc:240] Negotiation successful
0807 18:13:10.090384 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":196}
I20260807 18:13:10.091979 28809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:10.048692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32868 (local address 127.18.154.60:36639)
0807 18:13:10.049890 (+ 1198us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:10.049895 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:10.049917 (+ 22us) server_negotiation.cc:407] Connection header received
0807 18:13:10.050015 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:10.050020 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:10.050114 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:10.051325 (+ 1211us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:10.072452 (+ 21127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:10.074388 (+ 1936us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:10.076480 (+ 2092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:10.077382 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:10.080241 (+ 2859us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:10.081731 (+ 1490us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:13:10.081737 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:13:10.081740 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:13:10.081783 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:13:10.087170 (+ 5387us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:10.088182 (+ 1012us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:10.088189 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:10.088193 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:10.088428 (+ 235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:10.088946 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:10.088951 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:10.088954 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:10.089388 (+ 434us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:13:10.089551 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0807 18:13:10.091264 (+ 1713us) server_negotiation.cc:299] Negotiation successful
0807 18:13:10.091513 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1031,"thread_start_us":1282,"threads_started":1}
I20260807 18:13:10.093677 27766 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.18.154.60:36639: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "3d6d83d5ddcb4df3bcba443888204a34"
member_type: NON_VOTER
last_known_addr {
host: "127.18.154.60"
port: 36639
}
attrs {
promote: true
}
}
dest_uuid: "cc97b2cc29884038b29e0e18de9a6d18"
cas_config_opid_index: 6
I20260807 18:13:10.094466 27766 consensus_queue.cc:237] T 00000000000000000000000000000000 P cc97b2cc29884038b29e0e18de9a6d18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "cc97b2cc29884038b29e0e18de9a6d18" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 34661 } } peers { permanent_uuid: "a6983037814748628555659bcf04b165" member_type: VOTER last_known_addr { host: "127.18.154.61" port: 38131 } attrs { promote: false } } peers { permanent_uuid: "3d6d83d5ddcb4df3bcba443888204a34" member_type: NON_VOTER last_known_addr { host: "127.18.154.60" port: 36639 } attrs { promote: true } }
I20260807 18:13:10.096837 28611 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a6983037814748628555659bcf04b165 [term 1 FOLLOWER]: Refusing update from remote peer cc97b2cc29884038b29e0e18de9a6d18: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260807 18:13:10.097651 28719 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3d6d83d5ddcb4df3bcba443888204a34 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:13:10.097652 28748 consensus_queue.cc:1035] T 00000000000000000000000000000000 P cc97b2cc29884038b29e0e18de9a6d18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6983037814748628555659bcf04b165" member_type: VOTER last_known_addr { host: "127.18.154.61" port: 38131 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260807 18:13:10.101325 28719 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 3d6d83d5ddcb4df3bcba443888204a34 [term 1 FOLLOWER]: Refusing update from remote peer cc97b2cc29884038b29e0e18de9a6d18: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260807 18:13:10.102209 28748 consensus_queue.cc:1035] T 00000000000000000000000000000000 P cc97b2cc29884038b29e0e18de9a6d18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d6d83d5ddcb4df3bcba443888204a34" member_type: NON_VOTER last_known_addr { host: "127.18.154.60" port: 36639 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:13:10.104094 28748 raft_consensus.cc:2932] T 00000000000000000000000000000000 P cc97b2cc29884038b29e0e18de9a6d18 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 3d6d83d5ddcb4df3bcba443888204a34 (127.18.154.60) added. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "cc97b2cc29884038b29e0e18de9a6d18" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 34661 } } peers { permanent_uuid: "a6983037814748628555659bcf04b165" member_type: VOTER last_known_addr { host: "127.18.154.61" port: 38131 } attrs { promote: false } } peers { permanent_uuid: "3d6d83d5ddcb4df3bcba443888204a34" member_type: NON_VOTER last_known_addr { host: "127.18.154.60" port: 36639 } attrs { promote: true } } }
I20260807 18:13:10.106230 28611 raft_consensus.cc:2932] T 00000000000000000000000000000000 P a6983037814748628555659bcf04b165 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 3d6d83d5ddcb4df3bcba443888204a34 (127.18.154.60) added. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "cc97b2cc29884038b29e0e18de9a6d18" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 34661 } } peers { permanent_uuid: "a6983037814748628555659bcf04b165" member_type: VOTER last_known_addr { host: "127.18.154.61" port: 38131 } attrs { promote: false } } peers { permanent_uuid: "3d6d83d5ddcb4df3bcba443888204a34" member_type: NON_VOTER last_known_addr { host: "127.18.154.60" port: 36639 } attrs { promote: true } } }
I20260807 18:13:10.109560 28748 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.18.154.60:36639
I20260807 18:13:10.110380 28544 master.cc:537] Master@127.18.154.60:36639 shutting down...
I20260807 18:13:10.110306 28752 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc97b2cc29884038b29e0e18de9a6d18 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "cc97b2cc29884038b29e0e18de9a6d18" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "cc97b2cc29884038b29e0e18de9a6d18" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 34661 } } peers { permanent_uuid: "a6983037814748628555659bcf04b165" member_type: VOTER last_known_addr { host: "127.18.154.61" port: 38131 } attrs { promote: false } } peers { permanent_uuid: "3d6d83d5ddcb4df3bcba443888204a34" member_type: NON_VOTER last_known_addr { host: "127.18.154.60" port: 36639 } attrs { promote: true } } }
I20260807 18:13:10.110638 28752 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc97b2cc29884038b29e0e18de9a6d18 [sys.catalog]: This master's current role is: LEADER
I20260807 18:13:10.111598 28752 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc97b2cc29884038b29e0e18de9a6d18 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "cc97b2cc29884038b29e0e18de9a6d18" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "cc97b2cc29884038b29e0e18de9a6d18" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 34661 } } peers { permanent_uuid: "a6983037814748628555659bcf04b165" member_type: VOTER last_known_addr { host: "127.18.154.61" port: 38131 } attrs { promote: false } } peers { permanent_uuid: "3d6d83d5ddcb4df3bcba443888204a34" member_type: NON_VOTER last_known_addr { host: "127.18.154.60" port: 36639 } attrs { promote: true } } }
I20260807 18:13:10.111871 28752 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc97b2cc29884038b29e0e18de9a6d18 [sys.catalog]: This master's current role is: LEADER
W20260807 18:13:10.110282 28690 tablet.cc:2369] T 00000000000000000000000000000000 P 3d6d83d5ddcb4df3bcba443888204a34: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:13:10.115134 28648 sys_catalog.cc:455] T 00000000000000000000000000000000 P a6983037814748628555659bcf04b165 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "cc97b2cc29884038b29e0e18de9a6d18" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "cc97b2cc29884038b29e0e18de9a6d18" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 34661 } } peers { permanent_uuid: "a6983037814748628555659bcf04b165" member_type: VOTER last_known_addr { host: "127.18.154.61" port: 38131 } attrs { promote: false } } peers { permanent_uuid: "3d6d83d5ddcb4df3bcba443888204a34" member_type: NON_VOTER last_known_addr { host: "127.18.154.60" port: 36639 } attrs { promote: true } } }
I20260807 18:13:10.115430 28648 sys_catalog.cc:458] T 00000000000000000000000000000000 P a6983037814748628555659bcf04b165 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 18:13:10.127085 28719 raft_consensus.cc:858] T 00000000000000000000000000000000 P 3d6d83d5ddcb4df3bcba443888204a34 [term 1 NON_PARTICIPANT]: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc97b2cc29884038b29e0e18de9a6d18" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 34661 } } peers { permanent_uuid: "a6983037814748628555659bcf04b165" member_type: VOTER last_known_addr { host: "127.18.154.61" port: 38131 } } peers { permanent_uuid: "3d6d83d5ddcb4df3bcba443888204a34" member_type: VOTER last_known_addr { host: "127.18.154.60" port: 36639 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "cc97b2cc29884038b29e0e18de9a6d18" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 34661 } } peers { permanent_uuid: "a6983037814748628555659bcf04b165" member_type: NON_VOTER last_known_addr { host: "127.18.154.61" port: 38131 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "cc97b2cc29884038b29e0e18de9a6d18" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 34661 } } peers { permanent_uuid: "a6983037814748628555659bcf04b165" member_type: VOTER last_known_addr { host: "127.18.154.61" port: 38131 } attrs { promote: false } }
W20260807 18:13:10.127295 28719 raft_consensus.cc:1565] T 00000000000000000000000000000000 P 3d6d83d5ddcb4df3bcba443888204a34 [term 1 NON_PARTICIPANT]: Could not prepare op 'term: 1 index: 6' and following 1 ops. Status for this op: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc97b2cc29884038b29e0e18de9a6d18" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 34661 } } peers { permanent_uuid: "a6983037814748628555659bcf04b165" member_type: VOTER last_known_addr { host: "127.18.154.61" port: 38131 } } peers { permanent_uuid: "3d6d83d5ddcb4df3bcba443888204a34" member_type: VOTER last_known_addr { host: "127.18.154.60" port: 36639 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "cc97b2cc29884038b29e0e18de9a6d18" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 34661 } } peers { permanent_uuid: "a6983037814748628555659bcf04b165" member_type: NON_VOTER last_known_addr { host: "127.18.154.61" port: 38131 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "cc97b2cc29884038b29e0e18de9a6d18" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 34661 } } peers { permanent_uuid: "a6983037814748628555659bcf04b165" member_type: VOTER last_known_addr { host: "127.18.154.61" port: 38131 } attrs { promote: false } }
I20260807 18:13:10.140033 28719 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 3d6d83d5ddcb4df3bcba443888204a34 [term 1 NON_PARTICIPANT]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER a6983037814748628555659bcf04b165 (127.18.154.61) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "cc97b2cc29884038b29e0e18de9a6d18" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 34661 } } peers { permanent_uuid: "a6983037814748628555659bcf04b165" member_type: NON_VOTER last_known_addr { host: "127.18.154.61" port: 38131 } attrs { promote: true } } }
I20260807 18:13:10.145359 28777 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d6d83d5ddcb4df3bcba443888204a34 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cc97b2cc29884038b29e0e18de9a6d18. Latest consensus state: current_term: 1 leader_uuid: "cc97b2cc29884038b29e0e18de9a6d18" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "cc97b2cc29884038b29e0e18de9a6d18" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 34661 } } peers { permanent_uuid: "a6983037814748628555659bcf04b165" member_type: NON_VOTER last_known_addr { host: "127.18.154.61" port: 38131 } attrs { promote: true } } }
I20260807 18:13:10.145645 28777 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d6d83d5ddcb4df3bcba443888204a34 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260807 18:13:10.145871 28777 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d6d83d5ddcb4df3bcba443888204a34 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "cc97b2cc29884038b29e0e18de9a6d18" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "cc97b2cc29884038b29e0e18de9a6d18" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 34661 } } peers { permanent_uuid: "a6983037814748628555659bcf04b165" member_type: NON_VOTER last_known_addr { host: "127.18.154.61" port: 38131 } attrs { promote: true } } }
I20260807 18:13:10.146057 28777 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d6d83d5ddcb4df3bcba443888204a34 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260807 18:13:10.147063 28816 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d6d83d5ddcb4df3bcba443888204a34 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "cc97b2cc29884038b29e0e18de9a6d18" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "cc97b2cc29884038b29e0e18de9a6d18" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 34661 } } peers { permanent_uuid: "a6983037814748628555659bcf04b165" member_type: NON_VOTER last_known_addr { host: "127.18.154.61" port: 38131 } attrs { promote: true } } }
I20260807 18:13:10.147313 28816 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d6d83d5ddcb4df3bcba443888204a34 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260807 18:13:10.147871 28752 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc97b2cc29884038b29e0e18de9a6d18 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "cc97b2cc29884038b29e0e18de9a6d18" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "cc97b2cc29884038b29e0e18de9a6d18" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 34661 } } peers { permanent_uuid: "a6983037814748628555659bcf04b165" member_type: VOTER last_known_addr { host: "127.18.154.61" port: 38131 } attrs { promote: false } } peers { permanent_uuid: "3d6d83d5ddcb4df3bcba443888204a34" member_type: NON_VOTER last_known_addr { host: "127.18.154.60" port: 36639 } attrs { promote: true } } }
I20260807 18:13:10.148192 28752 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc97b2cc29884038b29e0e18de9a6d18 [sys.catalog]: This master's current role is: LEADER
W20260807 18:13:10.148828 27741 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
I20260807 18:13:10.149513 28544 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 3d6d83d5ddcb4df3bcba443888204a34 [term 1 NON_PARTICIPANT]: Raft consensus shutting down.
I20260807 18:13:10.150066 28544 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 3d6d83d5ddcb4df3bcba443888204a34 [term 1 NON_PARTICIPANT]: Raft consensus is shut down!
I20260807 18:13:10.150261 28544 tablet_replica.cc:330] T 00000000000000000000000000000000 P 3d6d83d5ddcb4df3bcba443888204a34: stopping tablet replica
W20260807 18:13:10.155401 27741 consensus_peers.cc:487] T 00000000000000000000000000000000 P cc97b2cc29884038b29e0e18de9a6d18 -> Peer 3d6d83d5ddcb4df3bcba443888204a34 (127.18.154.60:36639): Couldn't send request to peer 3d6d83d5ddcb4df3bcba443888204a34. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260807 18:13:10.159225 28544 master.cc:552] Master@127.18.154.60:36639 shutdown complete.
I20260807 18:13:10.159344 28544 master_runner.cc:305] Clearing existing system tablet
I20260807 18:13:10.159680 28544 fs_manager.cc:359] Metadata directory not provided
I20260807 18:13:10.159816 28544 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:13:10.162334 28544 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260807 18:13:10.165103 28824 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:13:10.165462 28544 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:13:10.165794 28544 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 18:13:10.165906 28544 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/master-2/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/master-2/wal
uuid: "3d6d83d5ddcb4df3bcba443888204a34"
format_stamp: "Formatted at 2026-08-07 18:13:09 on dist-test-slave-gkw7"
server_key: "621de20b74cb30499eabcb746c2cbeed"
server_key_iv: "27bd2600f132223f7b6f683df9faee12"
server_key_version: "encryptionkey@0"
I20260807 18:13:10.169457 28544 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 3d6d83d5ddcb4df3bcba443888204a34: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:13:10.174881 28544 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 3d6d83d5ddcb4df3bcba443888204a34: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260807 18:13:10.175076 28544 log.cc:1196] T 00000000000000000000000000000000 P 3d6d83d5ddcb4df3bcba443888204a34: Deleting WAL directory at /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260807 18:13:10.175628 28544 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 3d6d83d5ddcb4df3bcba443888204a34: Deleting consensus metadata
I20260807 18:13:10.175872 28544 master_runner.cc:315] Copying system tablet from 127.18.154.62:34661
I20260807 18:13:10.184870 28544 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 3d6d83d5ddcb4df3bcba443888204a34: tablet copy: Beginning tablet copy session from remote peer at address 127.18.154.62:34661
I20260807 18:13:10.207743 28771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:10.185781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52348 (local address 127.18.154.62:34661)
0807 18:13:10.186182 (+ 401us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:10.186191 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:10.194491 (+ 8300us) server_negotiation.cc:407] Connection header received
0807 18:13:10.194791 (+ 300us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:10.194796 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:10.194867 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:10.194984 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:10.196151 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:10.196941 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:10.198522 (+ 1581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:10.199096 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:10.199233 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:10.201600 (+ 2367us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:13:10.201606 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:13:10.201609 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:13:10.201681 (+ 72us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:13:10.204773 (+ 3092us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:10.205645 (+ 872us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:10.205652 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:10.205656 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:10.205805 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:10.206228 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:10.206232 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:10.206235 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:10.206519 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:13:10.206680 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0807 18:13:10.207104 (+ 424us) server_negotiation.cc:299] Negotiation successful
0807 18:13:10.207325 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":226}
I20260807 18:13:10.207840 28832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:10.185751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:34661 (local address 127.0.0.1:52348)
0807 18:13:10.194333 (+ 8582us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:10.194357 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 18:13:10.194604 (+ 247us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:13:10.195023 (+ 419us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:13:10.195028 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:13:10.195713 (+ 685us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:13:10.195955 (+ 242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:10.195965 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:10.197085 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:10.197092 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:10.198376 (+ 1284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:10.198390 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:10.199269 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:10.199277 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:10.199505 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:10.200171 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:13:10.200190 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:13:10.201346 (+ 1156us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:13:10.204909 (+ 3563us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:13:10.204914 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:13:10.204919 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:13:10.205469 (+ 550us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:13:10.205925 (+ 456us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:13:10.205930 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:13:10.205934 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:13:10.206099 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:13:10.206642 (+ 543us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:13:10.206647 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:13:10.206838 (+ 191us) client_negotiation.cc:769] Sending connection context
0807 18:13:10.206983 (+ 145us) client_negotiation.cc:240] Negotiation successful
0807 18:13:10.207325 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8432,"thread_start_us":8274,"threads_started":1}
I20260807 18:13:10.208690 27790 tablet_copy_service.cc:140] P cc97b2cc29884038b29e0e18de9a6d18: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 3d6d83d5ddcb4df3bcba443888204a34 ({username='oryx', principal='oryx/127.18.154.60@KRBTEST.COM'} at 127.0.0.1:52348)
I20260807 18:13:10.208884 27790 tablet_copy_service.cc:161] P cc97b2cc29884038b29e0e18de9a6d18: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 3d6d83d5ddcb4df3bcba443888204a34 at {username='oryx', principal='oryx/127.18.154.60@KRBTEST.COM'} at 127.0.0.1:52348: session id = 3d6d83d5ddcb4df3bcba443888204a34-00000000000000000000000000000000
I20260807 18:13:10.211257 27790 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P cc97b2cc29884038b29e0e18de9a6d18: Tablet Copy: opened 0 blocks and 1 log segments
I20260807 18:13:10.212949 28544 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260807 18:13:10.217989 28544 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 3d6d83d5ddcb4df3bcba443888204a34: tablet copy: Starting download of 0 data blocks...
I20260807 18:13:10.218297 28544 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 3d6d83d5ddcb4df3bcba443888204a34: tablet copy: Starting download of 1 WAL segments...
I20260807 18:13:10.220532 28544 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 3d6d83d5ddcb4df3bcba443888204a34: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260807 18:13:10.223971 27790 tablet_copy_service.cc:342] P cc97b2cc29884038b29e0e18de9a6d18: Request end of tablet copy session 3d6d83d5ddcb4df3bcba443888204a34-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.18.154.60@KRBTEST.COM'} at 127.0.0.1:52348
I20260807 18:13:10.224121 27790 tablet_copy_service.cc:433] P cc97b2cc29884038b29e0e18de9a6d18: ending tablet copy session 3d6d83d5ddcb4df3bcba443888204a34-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 3d6d83d5ddcb4df3bcba443888204a34
I20260807 18:13:10.235028 28544 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 18:13:10.238221 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 27856
W20260807 18:13:10.247287 28834 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:13:10.250406 28833 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:13:10.258540 28838 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:13:10.262995 28544 server_base.cc:1034] running on GCE node
I20260807 18:13:10.263424 28544 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:13:10.264178 28544 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:13:10.265424 28544 hybrid_clock.cc:648] HybridClock initialized: now 1786126390265362 us; error 46 us; skew 500 ppm
I20260807 18:13:10.265789 28544 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:13:10 dist-test-slave-gkw7 krb5kdc[27700](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126390, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.154.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:13:10.268265 28544 init.cc:377] Logged in from keytab as oryx/127.18.154.60@KRBTEST.COM (short username oryx)
I20260807 18:13:10.271730 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.154.1:41283
--local_ip_for_outbound_sockets=127.18.154.1
--tserver_master_addrs=127.18.154.62:34661
--webserver_port=37393
--webserver_interface=127.18.154.1
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/krb5kdc/oryx_127.18.154.1.keytab
--principal=oryx/127.18.154.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.154.20:38893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.18.154.62:34661,127.18.154.61:38131
--tserver_master_addrs=127.18.154.62:34661,127.18.154.61:38131,127.18.154.60:36639 with env {KRB5CCNAME=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 18:13:10.275909 28544 webserver.cc:460] Webserver started at http://127.18.154.60:42547/ using document root <none> and password file <none>
I20260807 18:13:10.276279 28544 fs_manager.cc:359] Metadata directory not provided
I20260807 18:13:10.276408 28544 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:13:10.279392 28544 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260807 18:13:10.281584 28850 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:13:10.282169 28544 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 18:13:10.282320 28544 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/master-2/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/master-2/wal
uuid: "3d6d83d5ddcb4df3bcba443888204a34"
format_stamp: "Formatted at 2026-08-07 18:13:09 on dist-test-slave-gkw7"
server_key: "621de20b74cb30499eabcb746c2cbeed"
server_key_iv: "27bd2600f132223f7b6f683df9faee12"
server_key_version: "encryptionkey@0"
I20260807 18:13:10.282536 28544 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:13:10.309440 28544 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:13:10.313763 28544 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:13:10.333787 28544 rpc_server.cc:307] RPC server started. Bound to: 127.18.154.60:36639
I20260807 18:13:10.333850 28920 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.154.60:36639 every 8 connection(s)
I20260807 18:13:10.335016 28544 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/master-2/data/info.pb
I20260807 18:13:10.340886 28923 sys_catalog.cc:263] Verifying existing consensus state
I20260807 18:13:10.344874 28923 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d6d83d5ddcb4df3bcba443888204a34: Bootstrap starting.
I20260807 18:13:10.366981 28923 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d6d83d5ddcb4df3bcba443888204a34: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 18:13:10.367549 28923 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d6d83d5ddcb4df3bcba443888204a34: Bootstrap complete.
I20260807 18:13:10.368714 28923 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3d6d83d5ddcb4df3bcba443888204a34 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "cc97b2cc29884038b29e0e18de9a6d18" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 34661 } } peers { permanent_uuid: "a6983037814748628555659bcf04b165" member_type: VOTER last_known_addr { host: "127.18.154.61" port: 38131 } attrs { promote: false } } peers { permanent_uuid: "3d6d83d5ddcb4df3bcba443888204a34" member_type: NON_VOTER last_known_addr { host: "127.18.154.60" port: 36639 } attrs { promote: true } }
I20260807 18:13:10.369043 28923 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3d6d83d5ddcb4df3bcba443888204a34 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 3d6d83d5ddcb4df3bcba443888204a34, State: Initialized, Role: LEARNER
I20260807 18:13:10.369323 28923 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3d6d83d5ddcb4df3bcba443888204a34 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "cc97b2cc29884038b29e0e18de9a6d18" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 34661 } } peers { permanent_uuid: "a6983037814748628555659bcf04b165" member_type: VOTER last_known_addr { host: "127.18.154.61" port: 38131 } attrs { promote: false } } peers { permanent_uuid: "3d6d83d5ddcb4df3bcba443888204a34" member_type: NON_VOTER last_known_addr { host: "127.18.154.60" port: 36639 } attrs { promote: true } }
I20260807 18:13:10.370278 28927 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d6d83d5ddcb4df3bcba443888204a34 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "cc97b2cc29884038b29e0e18de9a6d18" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 34661 } } peers { permanent_uuid: "a6983037814748628555659bcf04b165" member_type: VOTER last_known_addr { host: "127.18.154.61" port: 38131 } attrs { promote: false } } peers { permanent_uuid: "3d6d83d5ddcb4df3bcba443888204a34" member_type: NON_VOTER last_known_addr { host: "127.18.154.60" port: 36639 } attrs { promote: true } } }
I20260807 18:13:10.370687 28923 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3d6d83d5ddcb4df3bcba443888204a34 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:13:10.370755 28927 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d6d83d5ddcb4df3bcba443888204a34 [sys.catalog]: This master's current role is: LEARNER
I20260807 18:13:10.381052 28943 catalog_manager.cc:1259] Loaded cluster ID: 6f80460f0d5e43d69126321a9a079ffa
I20260807 18:13:10.381189 28943 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 3d6d83d5ddcb4df3bcba443888204a34: loading cluster ID for follower catalog manager: success
I20260807 18:13:10.384837 28943 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 3d6d83d5ddcb4df3bcba443888204a34: acquiring CA information for follower catalog manager: success
I20260807 18:13:10.387408 28943 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 3d6d83d5ddcb4df3bcba443888204a34: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 18:13:10.549582 28752 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc97b2cc29884038b29e0e18de9a6d18 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "cc97b2cc29884038b29e0e18de9a6d18" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "cc97b2cc29884038b29e0e18de9a6d18" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 34661 } } peers { permanent_uuid: "a6983037814748628555659bcf04b165" member_type: VOTER last_known_addr { host: "127.18.154.61" port: 38131 } attrs { promote: false } } peers { permanent_uuid: "3d6d83d5ddcb4df3bcba443888204a34" member_type: NON_VOTER last_known_addr { host: "127.18.154.60" port: 36639 } attrs { promote: true } } }
I20260807 18:13:10.550050 28752 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc97b2cc29884038b29e0e18de9a6d18 [sys.catalog]: This master's current role is: LEADER
I20260807 18:13:10.557453 28944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:10.550263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32884 (local address 127.18.154.60:36639)
0807 18:13:10.551030 (+ 767us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:10.551036 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:10.551059 (+ 23us) server_negotiation.cc:407] Connection header received
0807 18:13:10.551128 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:10.551133 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:10.551208 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:10.551400 (+ 192us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:13:10.551961 (+ 561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:10.553079 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:10.555395 (+ 2316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:10.556230 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:10.556346 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:10.556529 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0807 18:13:10.556946 (+ 417us) server_negotiation.cc:299] Negotiation successful
0807 18:13:10.557118 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":579,"thread_start_us":404,"threads_started":1}
I20260807 18:13:10.557473 28762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:10.550052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.60:36639 (local address 127.0.0.1:32884)
0807 18:13:10.550358 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:10.550383 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0807 18:13:10.550536 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:13:10.551439 (+ 903us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:13:10.551448 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:13:10.551491 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:13:10.551752 (+ 261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:10.551767 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:10.553240 (+ 1473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:10.553247 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:10.555141 (+ 1894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:10.555161 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:10.556426 (+ 1265us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:10.556435 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:10.556695 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:10.556751 (+ 56us) client_negotiation.cc:769] Sending connection context
0807 18:13:10.556860 (+ 109us) client_negotiation.cc:240] Negotiation successful
0807 18:13:10.557065 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150}
I20260807 18:13:10.558401 28886 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 3d6d83d5ddcb4df3bcba443888204a34 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.5->[1.6-1.7] Dedup: 1.7->[]
I20260807 18:13:10.558898 28927 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d6d83d5ddcb4df3bcba443888204a34 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cc97b2cc29884038b29e0e18de9a6d18. Latest consensus state: current_term: 1 leader_uuid: "cc97b2cc29884038b29e0e18de9a6d18" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "cc97b2cc29884038b29e0e18de9a6d18" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 34661 } } peers { permanent_uuid: "a6983037814748628555659bcf04b165" member_type: VOTER last_known_addr { host: "127.18.154.61" port: 38131 } attrs { promote: false } } peers { permanent_uuid: "3d6d83d5ddcb4df3bcba443888204a34" member_type: NON_VOTER last_known_addr { host: "127.18.154.60" port: 36639 } attrs { promote: true } } }
I20260807 18:13:10.559209 28927 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d6d83d5ddcb4df3bcba443888204a34 [sys.catalog]: This master's current role is: LEARNER
I20260807 18:13:10.559643 28752 raft_consensus.cc:1050] T 00000000000000000000000000000000 P cc97b2cc29884038b29e0e18de9a6d18: attempting to promote NON_VOTER 3d6d83d5ddcb4df3bcba443888204a34 to VOTER
I20260807 18:13:10.560302 28752 consensus_queue.cc:237] T 00000000000000000000000000000000 P cc97b2cc29884038b29e0e18de9a6d18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "cc97b2cc29884038b29e0e18de9a6d18" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 34661 } } peers { permanent_uuid: "a6983037814748628555659bcf04b165" member_type: VOTER last_known_addr { host: "127.18.154.61" port: 38131 } attrs { promote: false } } peers { permanent_uuid: "3d6d83d5ddcb4df3bcba443888204a34" member_type: VOTER last_known_addr { host: "127.18.154.60" port: 36639 } attrs { promote: false } }
I20260807 18:13:10.562212 28886 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 3d6d83d5ddcb4df3bcba443888204a34 [term 1 LEARNER]: Refusing update from remote peer cc97b2cc29884038b29e0e18de9a6d18: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260807 18:13:10.562433 28611 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a6983037814748628555659bcf04b165 [term 1 FOLLOWER]: Refusing update from remote peer cc97b2cc29884038b29e0e18de9a6d18: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260807 18:13:10.563011 28748 consensus_queue.cc:1035] T 00000000000000000000000000000000 P cc97b2cc29884038b29e0e18de9a6d18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d6d83d5ddcb4df3bcba443888204a34" member_type: VOTER last_known_addr { host: "127.18.154.60" port: 36639 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260807 18:13:10.563578 28759 consensus_queue.cc:1035] T 00000000000000000000000000000000 P cc97b2cc29884038b29e0e18de9a6d18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6983037814748628555659bcf04b165" member_type: VOTER last_known_addr { host: "127.18.154.61" port: 38131 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260807 18:13:10.566159 28748 raft_consensus.cc:2932] T 00000000000000000000000000000000 P cc97b2cc29884038b29e0e18de9a6d18 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 3d6d83d5ddcb4df3bcba443888204a34 (127.18.154.60) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "cc97b2cc29884038b29e0e18de9a6d18" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 34661 } } peers { permanent_uuid: "a6983037814748628555659bcf04b165" member_type: VOTER last_known_addr { host: "127.18.154.61" port: 38131 } attrs { promote: false } } peers { permanent_uuid: "3d6d83d5ddcb4df3bcba443888204a34" member_type: VOTER last_known_addr { host: "127.18.154.60" port: 36639 } attrs { promote: false } } }
I20260807 18:13:10.566864 28611 raft_consensus.cc:2932] T 00000000000000000000000000000000 P a6983037814748628555659bcf04b165 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 3d6d83d5ddcb4df3bcba443888204a34 (127.18.154.60) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "cc97b2cc29884038b29e0e18de9a6d18" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 34661 } } peers { permanent_uuid: "a6983037814748628555659bcf04b165" member_type: VOTER last_known_addr { host: "127.18.154.61" port: 38131 } attrs { promote: false } } peers { permanent_uuid: "3d6d83d5ddcb4df3bcba443888204a34" member_type: VOTER last_known_addr { host: "127.18.154.60" port: 36639 } attrs { promote: false } } }
I20260807 18:13:10.571246 28752 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc97b2cc29884038b29e0e18de9a6d18 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "cc97b2cc29884038b29e0e18de9a6d18" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "cc97b2cc29884038b29e0e18de9a6d18" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 34661 } } peers { permanent_uuid: "a6983037814748628555659bcf04b165" member_type: VOTER last_known_addr { host: "127.18.154.61" port: 38131 } attrs { promote: false } } peers { permanent_uuid: "3d6d83d5ddcb4df3bcba443888204a34" member_type: VOTER last_known_addr { host: "127.18.154.60" port: 36639 } attrs { promote: false } } }
I20260807 18:13:10.571576 28752 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc97b2cc29884038b29e0e18de9a6d18 [sys.catalog]: This master's current role is: LEADER
I20260807 18:13:10.571406 28748 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc97b2cc29884038b29e0e18de9a6d18 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "cc97b2cc29884038b29e0e18de9a6d18" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "cc97b2cc29884038b29e0e18de9a6d18" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 34661 } } peers { permanent_uuid: "a6983037814748628555659bcf04b165" member_type: VOTER last_known_addr { host: "127.18.154.61" port: 38131 } attrs { promote: false } } peers { permanent_uuid: "3d6d83d5ddcb4df3bcba443888204a34" member_type: VOTER last_known_addr { host: "127.18.154.60" port: 36639 } attrs { promote: false } } }
I20260807 18:13:10.571712 28748 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc97b2cc29884038b29e0e18de9a6d18 [sys.catalog]: This master's current role is: LEADER
I20260807 18:13:10.572614 28648 sys_catalog.cc:455] T 00000000000000000000000000000000 P a6983037814748628555659bcf04b165 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "cc97b2cc29884038b29e0e18de9a6d18" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "cc97b2cc29884038b29e0e18de9a6d18" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 34661 } } peers { permanent_uuid: "a6983037814748628555659bcf04b165" member_type: VOTER last_known_addr { host: "127.18.154.61" port: 38131 } attrs { promote: false } } peers { permanent_uuid: "3d6d83d5ddcb4df3bcba443888204a34" member_type: VOTER last_known_addr { host: "127.18.154.60" port: 36639 } attrs { promote: false } } }
I20260807 18:13:10.572883 28648 sys_catalog.cc:458] T 00000000000000000000000000000000 P a6983037814748628555659bcf04b165 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 18:13:10.579216 28752 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc97b2cc29884038b29e0e18de9a6d18 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "cc97b2cc29884038b29e0e18de9a6d18" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "cc97b2cc29884038b29e0e18de9a6d18" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 34661 } } peers { permanent_uuid: "a6983037814748628555659bcf04b165" member_type: VOTER last_known_addr { host: "127.18.154.61" port: 38131 } attrs { promote: false } } peers { permanent_uuid: "3d6d83d5ddcb4df3bcba443888204a34" member_type: VOTER last_known_addr { host: "127.18.154.60" port: 36639 } attrs { promote: false } } }
I20260807 18:13:10.579514 28752 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc97b2cc29884038b29e0e18de9a6d18 [sys.catalog]: This master's current role is: LEADER
I20260807 18:13:10.580003 28886 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 3d6d83d5ddcb4df3bcba443888204a34 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 3d6d83d5ddcb4df3bcba443888204a34 (127.18.154.60) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "cc97b2cc29884038b29e0e18de9a6d18" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 34661 } } peers { permanent_uuid: "a6983037814748628555659bcf04b165" member_type: VOTER last_known_addr { host: "127.18.154.61" port: 38131 } attrs { promote: false } } peers { permanent_uuid: "3d6d83d5ddcb4df3bcba443888204a34" member_type: VOTER last_known_addr { host: "127.18.154.60" port: 36639 } attrs { promote: false } } }
I20260807 18:13:10.583217 28927 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d6d83d5ddcb4df3bcba443888204a34 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "cc97b2cc29884038b29e0e18de9a6d18" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "cc97b2cc29884038b29e0e18de9a6d18" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 34661 } } peers { permanent_uuid: "a6983037814748628555659bcf04b165" member_type: VOTER last_known_addr { host: "127.18.154.61" port: 38131 } attrs { promote: false } } peers { permanent_uuid: "3d6d83d5ddcb4df3bcba443888204a34" member_type: VOTER last_known_addr { host: "127.18.154.60" port: 36639 } attrs { promote: false } } }
I20260807 18:13:10.583479 28927 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d6d83d5ddcb4df3bcba443888204a34 [sys.catalog]: This master's current role is: FOLLOWER
W20260807 18:13:10.676255 28849 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:13:10.676571 28849 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:13:10.676774 28849 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:13:10.685200 28849 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:13:10.685331 28849 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:13:10.685508 28849 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.154.1
I20260807 18:13:10.696237 28849 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.154.20:38893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/krb5kdc/oryx_127.18.154.1.keytab
--principal=oryx/127.18.154.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.154.1:41283
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.154.1
--webserver_port=37393
--tserver_master_addrs=127.18.154.62:34661,127.18.154.61:38131,127.18.154.60:36639
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.154.1
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:13:10.697728 28849 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:13:10.699127 28849 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:13:10.713927 28954 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:13:10.714048 28958 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:13:10.714087 28953 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:13:10.715068 28849 server_base.cc:1034] running on GCE node
I20260807 18:13:10.715754 28849 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:13:10.716823 28849 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:13:10.718068 28849 hybrid_clock.cc:648] HybridClock initialized: now 1786126390717995 us; error 59 us; skew 500 ppm
I20260807 18:13:10.718443 28849 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:13:10 dist-test-slave-gkw7 krb5kdc[27700](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126390, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.154.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:13:10.722048 28849 init.cc:377] Logged in from keytab as oryx/127.18.154.1@KRBTEST.COM (short username oryx)
I20260807 18:13:10.723871 28849 webserver.cc:460] Webserver started at http://127.18.154.1:37393/ using document root <none> and password file <none>
I20260807 18:13:10.724762 28849 fs_manager.cc:359] Metadata directory not provided
I20260807 18:13:10.724975 28849 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:13:10.731351 28849 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 18:13:10.734550 28967 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:13:10.736176 28849 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 18:13:10.736328 28849 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/ts-0/wal
uuid: "3d0c878c4dd74021bd3406b93a11c7c4"
format_stamp: "Formatted at 2026-08-07 18:13:07 on dist-test-slave-gkw7"
server_key: "7e8cee6f20f17ebf0fad660de646409d"
server_key_iv: "35b8334b13fd23651f4c9e9af667bde6"
server_key_version: "encryptionkey@0"
I20260807 18:13:10.737399 28849 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:13:10.757062 28849 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:13:10.761310 28849 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:13:10.761742 28849 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:13:10.762777 28849 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:13:10.764259 28849 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:13:10.764348 28849 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:13:10.764437 28849 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:13:10.764494 28849 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:13:10.810504 28849 rpc_server.cc:307] RPC server started. Bound to: 127.18.154.1:41283
I20260807 18:13:10.810741 29131 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.154.1:41283 every 8 connection(s)
I20260807 18:13:10.812084 28849 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/ts-0/data/info.pb
I20260807 18:13:10.815451 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 28849
I20260807 18:13:10.815876 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 28036
Aug 07 18:13:10 dist-test-slave-gkw7 krb5kdc[27700](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126390, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.154.1@KRBTEST.COM for oryx/127.18.154.60@KRBTEST.COM
Aug 07 18:13:10 dist-test-slave-gkw7 krb5kdc[27700](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126390, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.154.1@KRBTEST.COM for oryx/127.18.154.62@KRBTEST.COM
Aug 07 18:13:10 dist-test-slave-gkw7 krb5kdc[27700](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126390, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.154.1@KRBTEST.COM for oryx/127.18.154.61@KRBTEST.COM
Aug 07 18:13:10 dist-test-slave-gkw7 krb5kdc[27700](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126386, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.18.154.2@KRBTEST.COM
I20260807 18:13:10.857797 28944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:10.833909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.1:56063 (local address 127.18.154.60:36639)
0807 18:13:10.834124 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:10.834131 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:10.835164 (+ 1033us) server_negotiation.cc:407] Connection header received
0807 18:13:10.835305 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:10.835310 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:10.835389 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:10.835485 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:10.837987 (+ 2502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:10.839153 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:10.841177 (+ 2024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:10.841888 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:10.842000 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:10.848953 (+ 6953us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:13:10.848958 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:13:10.848962 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:13:10.848998 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:13:10.852728 (+ 3730us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:10.854148 (+ 1420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:10.854155 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:10.854162 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:10.854338 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:10.854922 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:10.854929 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:10.854932 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:10.855378 (+ 446us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:13:10.855517 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0807 18:13:10.857070 (+ 1553us) server_negotiation.cc:299] Negotiation successful
0807 18:13:10.857374 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260807 18:13:10.859228 29136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:10.815461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.1:38283 (local address 127.18.154.62:34661)
0807 18:13:10.816472 (+ 1011us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:10.816481 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:10.828785 (+ 12304us) server_negotiation.cc:407] Connection header received
0807 18:13:10.834530 (+ 5745us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:10.834541 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:10.834678 (+ 137us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:10.834811 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:10.837959 (+ 3148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:10.839153 (+ 1194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:10.843039 (+ 3886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:10.843739 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:10.843882 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:10.850527 (+ 6645us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:13:10.850554 (+ 27us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:13:10.850583 (+ 29us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:13:10.850638 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:13:10.855136 (+ 4498us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:10.856530 (+ 1394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:10.856539 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:10.856543 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:10.856720 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:10.857219 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:10.857224 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:10.857228 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:10.857624 (+ 396us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:13:10.857784 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0807 18:13:10.858628 (+ 844us) server_negotiation.cc:299] Negotiation successful
0807 18:13:10.858905 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":830,"thread_start_us":638,"threads_started":1}
I20260807 18:13:10.860755 29139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:10.816800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.61:38131 (local address 127.18.154.1:56183)
0807 18:13:10.833419 (+ 16619us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:10.833438 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 18:13:10.835049 (+ 1611us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:13:10.835708 (+ 659us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:13:10.835711 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:13:10.836761 (+ 1050us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:13:10.837784 (+ 1023us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:10.837801 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:10.840024 (+ 2223us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:10.840029 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:10.842751 (+ 2722us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:10.842766 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:10.845297 (+ 2531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:10.845305 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:10.845480 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:10.846193 (+ 713us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:13:10.846225 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:13:10.851167 (+ 4942us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:13:10.855627 (+ 4460us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:13:10.855634 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:13:10.855638 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:13:10.856408 (+ 770us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:13:10.856897 (+ 489us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:13:10.856904 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:13:10.856907 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:13:10.857085 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:13:10.857808 (+ 723us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:13:10.857816 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:13:10.858743 (+ 927us) client_negotiation.cc:769] Sending connection context
0807 18:13:10.858896 (+ 153us) client_negotiation.cc:240] Negotiation successful
0807 18:13:10.859188 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16475,"mutex_wait_us":1025,"spinlock_wait_cycles":3840,"thread_start_us":11001,"threads_started":1}
I20260807 18:13:10.860877 29146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:10.817643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37858 (local address 127.18.154.2:42923)
0807 18:13:10.834286 (+ 16643us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:10.834295 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:10.834329 (+ 34us) server_negotiation.cc:407] Connection header received
0807 18:13:10.835475 (+ 1146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:10.835489 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:10.835832 (+ 343us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:10.835992 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:10.838484 (+ 2492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:10.840171 (+ 1687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:10.842437 (+ 2266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:10.843341 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:10.843488 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:10.851816 (+ 8328us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:13:10.851855 (+ 39us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:13:10.851893 (+ 38us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:13:10.851964 (+ 71us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:13:10.856788 (+ 4824us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:10.857815 (+ 1027us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:10.857824 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:10.857827 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:10.858059 (+ 232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:10.858525 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:10.858530 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:10.858533 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:10.858978 (+ 445us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:13:10.859310 (+ 332us) server_negotiation.cc:1035] Waiting for connection context
0807 18:13:10.860013 (+ 703us) server_negotiation.cc:299] Negotiation successful
0807 18:13:10.860318 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16453,"thread_start_us":16694,"threads_started":1}
I20260807 18:13:10.861286 29145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:10.817410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.1:56183 (local address 127.18.154.61:38131)
0807 18:13:10.827410 (+ 10000us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:10.827420 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:10.833601 (+ 6181us) server_negotiation.cc:407] Connection header received
0807 18:13:10.835287 (+ 1686us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:10.835292 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:10.835389 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:10.835485 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:10.838645 (+ 3160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:10.839814 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:10.842922 (+ 3108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:10.845014 (+ 2092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:10.845150 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:10.851399 (+ 6249us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:13:10.851408 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:13:10.851414 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:13:10.851458 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:13:10.855495 (+ 4037us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:10.856555 (+ 1060us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:10.856561 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:10.856564 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:10.856754 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:10.857217 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:10.857222 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:10.857226 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:10.857592 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:13:10.857737 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0807 18:13:10.860508 (+ 2771us) server_negotiation.cc:299] Negotiation successful
0807 18:13:10.860821 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9729,"thread_start_us":10030,"threads_started":1}
I20260807 18:13:10.860826 29142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:10.816372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:34661 (local address 127.18.154.1:38283)
0807 18:13:10.828475 (+ 12103us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:10.828556 (+ 81us) client_negotiation.cc:174] Beginning negotiation
0807 18:13:10.832477 (+ 3921us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:13:10.835181 (+ 2704us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:13:10.835208 (+ 27us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:13:10.836407 (+ 1199us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:13:10.837780 (+ 1373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:10.837807 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:10.840868 (+ 3061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:10.840873 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:10.842903 (+ 2030us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:10.842920 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:10.843927 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:10.843932 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:10.844048 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:10.844610 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:13:10.844628 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:13:10.850270 (+ 5642us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:13:10.855583 (+ 5313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:13:10.855589 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:13:10.855595 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:13:10.856377 (+ 782us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:13:10.856876 (+ 499us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:13:10.856880 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:13:10.856884 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:13:10.857078 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:13:10.857867 (+ 789us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:13:10.857874 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:13:10.858322 (+ 448us) client_negotiation.cc:769] Sending connection context
0807 18:13:10.858471 (+ 149us) client_negotiation.cc:240] Negotiation successful
0807 18:13:10.859693 (+ 1222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11614,"mutex_wait_us":1162,"thread_start_us":16623,"threads_started":1}
I20260807 18:13:10.862138 29147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:10.833723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.60:36639 (local address 127.18.154.1:56063)
0807 18:13:10.835029 (+ 1306us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:10.835046 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 18:13:10.835176 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:13:10.835680 (+ 504us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:13:10.835686 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:13:10.836761 (+ 1075us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:13:10.837771 (+ 1010us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:10.837801 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:10.839334 (+ 1533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:10.839341 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:10.840861 (+ 1520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:10.840877 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:10.842396 (+ 1519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:10.842404 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:10.842883 (+ 479us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:10.844376 (+ 1493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:13:10.844437 (+ 61us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:13:10.848707 (+ 4270us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:13:10.852979 (+ 4272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:13:10.852992 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:13:10.853013 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:13:10.853989 (+ 976us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:13:10.854506 (+ 517us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:13:10.854512 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:13:10.854518 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:13:10.854782 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:13:10.856086 (+ 1304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:13:10.856094 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:13:10.856581 (+ 487us) client_negotiation.cc:769] Sending connection context
0807 18:13:10.858161 (+ 1580us) client_negotiation.cc:240] Negotiation successful
0807 18:13:10.858932 (+ 771us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1185,"mutex_wait_us":250,"thread_start_us":1008,"threads_started":1}
I20260807 18:13:10.863415 29134 heartbeater.cc:347] Connected to a master server at 127.18.154.61:38131
I20260807 18:13:10.863509 29133 heartbeater.cc:347] Connected to a master server at 127.18.154.62:34661
I20260807 18:13:10.863606 29132 heartbeater.cc:347] Connected to a master server at 127.18.154.60:36639
I20260807 18:13:10.863775 28216 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:13:10.863868 29134 heartbeater.cc:464] Registering TS with master...
I20260807 18:13:10.863914 29132 heartbeater.cc:464] Registering TS with master...
I20260807 18:13:10.863914 29133 heartbeater.cc:464] Registering TS with master...
I20260807 18:13:10.865512 29134 heartbeater.cc:511] Master 127.18.154.61:38131 requested a full tablet report, sending...
I20260807 18:13:10.865713 29132 heartbeater.cc:511] Master 127.18.154.60:36639 requested a full tablet report, sending...
I20260807 18:13:10.865527 29133 heartbeater.cc:511] Master 127.18.154.62:34661 requested a full tablet report, sending...
I20260807 18:13:10.867590 27766 ts_manager.cc:194] Re-registered known tserver with Master: 3d0c878c4dd74021bd3406b93a11c7c4 (127.18.154.1:41283)
I20260807 18:13:10.868467 27766 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.18.154.1@KRBTEST.COM'} at 127.18.154.1:38283
I20260807 18:13:10.868546 28874 ts_manager.cc:194] Registered new tserver with Master: 3d0c878c4dd74021bd3406b93a11c7c4 (127.18.154.1:41283)
I20260807 18:13:10.869899 28595 ts_manager.cc:194] Registered new tserver with Master: 3d0c878c4dd74021bd3406b93a11c7c4 (127.18.154.1:41283)
I20260807 18:13:10.956471 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 28036
I20260807 18:13:10.983385 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.154.2:42923
--local_ip_for_outbound_sockets=127.18.154.2
--tserver_master_addrs=127.18.154.62:34661
--webserver_port=34707
--webserver_interface=127.18.154.2
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/krb5kdc/oryx_127.18.154.2.keytab
--principal=oryx/127.18.154.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.154.20:38893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.18.154.62:34661,127.18.154.61:38131
--tserver_master_addrs=127.18.154.62:34661,127.18.154.61:38131,127.18.154.60:36639 with env {KRB5CCNAME=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:13:11.347036 29153 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:13:11.347419 29153 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:13:11.347563 29153 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:13:11.356900 29153 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:13:11.357057 29153 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:13:11.357430 29153 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.154.2
I20260807 18:13:11.368335 29153 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.154.20:38893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/krb5kdc/oryx_127.18.154.2.keytab
--principal=oryx/127.18.154.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.154.2:42923
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.154.2
--webserver_port=34707
--tserver_master_addrs=127.18.154.62:34661,127.18.154.61:38131,127.18.154.60:36639
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.154.2
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:13:11.369864 29153 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:13:11.371454 29153 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:13:11.386436 29160 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:13:11.386510 29162 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:13:11.387276 29159 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:13:11.387332 29153 server_base.cc:1034] running on GCE node
I20260807 18:13:11.388055 29153 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:13:11.389070 29153 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:13:11.390302 29153 hybrid_clock.cc:648] HybridClock initialized: now 1786126391390240 us; error 75 us; skew 500 ppm
I20260807 18:13:11.390643 29153 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:13:11 dist-test-slave-gkw7 krb5kdc[27700](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126391, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.154.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:13:11.393854 29153 init.cc:377] Logged in from keytab as oryx/127.18.154.2@KRBTEST.COM (short username oryx)
I20260807 18:13:11.395395 29153 webserver.cc:460] Webserver started at http://127.18.154.2:34707/ using document root <none> and password file <none>
I20260807 18:13:11.396273 29153 fs_manager.cc:359] Metadata directory not provided
I20260807 18:13:11.396425 29153 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:13:11.402385 29153 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260807 18:13:11.405375 29174 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:13:11.406896 29153 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:13:11.407085 29153 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/ts-1/wal
uuid: "fde1306083f041638796f175e031fb30"
format_stamp: "Formatted at 2026-08-07 18:13:07 on dist-test-slave-gkw7"
server_key: "ddd3048d1aacb8b1960999022f2ccca5"
server_key_iv: "f67d048bda83e2248ced505d2aa5adeb"
server_key_version: "encryptionkey@0"
I20260807 18:13:11.408118 29153 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:13:11.446555 29153 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:13:11.451500 29153 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:13:11.451785 29153 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:13:11.452783 29153 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:13:11.454367 29153 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:13:11.454459 29153 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:13:11.454598 29153 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:13:11.454680 29153 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:13:11.491325 29153 rpc_server.cc:307] RPC server started. Bound to: 127.18.154.2:42923
I20260807 18:13:11.491454 29328 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.154.2:42923 every 8 connection(s)
I20260807 18:13:11.492899 29153 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/ts-1/data/info.pb
I20260807 18:13:11.501040 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 29153
I20260807 18:13:11.501386 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 28228
Aug 07 18:13:11 dist-test-slave-gkw7 krb5kdc[27700](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126391, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.154.2@KRBTEST.COM for oryx/127.18.154.60@KRBTEST.COM
Aug 07 18:13:11 dist-test-slave-gkw7 krb5kdc[27700](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126386, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.18.154.3@KRBTEST.COM
Aug 07 18:13:11 dist-test-slave-gkw7 krb5kdc[27700](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126391, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.154.2@KRBTEST.COM for oryx/127.18.154.61@KRBTEST.COM
I20260807 18:13:11.546041 29340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:11.496215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.2:38119 (local address 127.18.154.61:38131)
0807 18:13:11.515851 (+ 19636us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:11.515858 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:11.517282 (+ 1424us) server_negotiation.cc:407] Connection header received
0807 18:13:11.523925 (+ 6643us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:11.523933 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:11.524035 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:11.524111 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:11.527163 (+ 3052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:11.528153 (+ 990us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:11.530145 (+ 1992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:11.531076 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:11.531248 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:11.537520 (+ 6272us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:13:11.537552 (+ 32us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:13:11.537558 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:13:11.537610 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:13:11.542004 (+ 4394us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:11.543180 (+ 1176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:11.543185 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:11.543189 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:11.543327 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:11.543834 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:11.543840 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:11.543857 (+ 17us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:11.544299 (+ 442us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:13:11.544443 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0807 18:13:11.545372 (+ 929us) server_negotiation.cc:299] Negotiation successful
0807 18:13:11.545747 (+ 375us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19478,"thread_start_us":19396,"threads_started":1}
Aug 07 18:13:11 dist-test-slave-gkw7 krb5kdc[27700](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126391, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.154.2@KRBTEST.COM for oryx/127.18.154.62@KRBTEST.COM
I20260807 18:13:11.546576 29338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:11.495905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.2:57325 (local address 127.18.154.60:36639)
0807 18:13:11.516256 (+ 20351us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:11.516263 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:11.516285 (+ 22us) server_negotiation.cc:407] Connection header received
0807 18:13:11.516349 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:11.516355 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:11.516442 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:11.516553 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:11.519589 (+ 3036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:11.521213 (+ 1624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:11.524531 (+ 3318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:11.525367 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:11.525474 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:11.531067 (+ 5593us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:13:11.531076 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:13:11.531081 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:13:11.531170 (+ 89us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:13:11.535584 (+ 4414us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:11.542521 (+ 6937us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:11.542527 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:11.542547 (+ 20us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:11.542753 (+ 206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:11.543456 (+ 703us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:11.543462 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:11.543465 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:11.543892 (+ 427us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:13:11.544037 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0807 18:13:11.545338 (+ 1301us) server_negotiation.cc:299] Negotiation successful
0807 18:13:11.545894 (+ 556us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20224,"thread_start_us":20128,"threads_started":1}
I20260807 18:13:11.548851 29341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:11.496696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.61:38131 (local address 127.18.154.2:38119)
0807 18:13:11.517170 (+ 20474us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:11.517195 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0807 18:13:11.523535 (+ 6340us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:13:11.524361 (+ 826us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:13:11.524366 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:13:11.525720 (+ 1354us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:13:11.526072 (+ 352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:11.526085 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:11.528360 (+ 2275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:11.528364 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:11.529987 (+ 1623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:11.530001 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:11.531581 (+ 1580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:11.531590 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:11.531833 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:11.533048 (+ 1215us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:13:11.533080 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:13:11.537262 (+ 4182us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:13:11.542156 (+ 4894us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:13:11.542162 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:13:11.542166 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:13:11.543038 (+ 872us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:13:11.543446 (+ 408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:13:11.543452 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:13:11.543458 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:13:11.543634 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:13:11.544529 (+ 895us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:13:11.544534 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:13:11.544811 (+ 277us) client_negotiation.cc:769] Sending connection context
0807 18:13:11.546380 (+ 1569us) client_negotiation.cc:240] Negotiation successful
0807 18:13:11.546997 (+ 617us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20303,"mutex_wait_us":40,"spinlock_wait_cycles":23040,"thread_start_us":17000,"threads_started":1}
I20260807 18:13:11.549191 29344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:11.503323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35968 (local address 127.18.154.3:46263)
0807 18:13:11.504227 (+ 904us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:11.504241 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:11.517607 (+ 13366us) server_negotiation.cc:407] Connection header received
0807 18:13:11.517699 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:11.517711 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:11.518037 (+ 326us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:11.521186 (+ 3149us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:11.521553 (+ 367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:11.523661 (+ 2108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:11.526756 (+ 3095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:11.527696 (+ 940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:11.528468 (+ 772us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:11.535035 (+ 6567us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:13:11.535058 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:13:11.535080 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:13:11.535155 (+ 75us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:13:11.542652 (+ 7497us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:11.543997 (+ 1345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:11.544007 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:11.544010 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:11.544233 (+ 223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:11.544925 (+ 692us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:11.544929 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:11.544932 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:11.545432 (+ 500us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:13:11.547978 (+ 2546us) server_negotiation.cc:1035] Waiting for connection context
0807 18:13:11.548264 (+ 286us) server_negotiation.cc:299] Negotiation successful
0807 18:13:11.548677 (+ 413us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":621,"thread_start_us":423,"threads_started":1}
I20260807 18:13:11.549907 29342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:11.496695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.60:36639 (local address 127.18.154.2:57325)
0807 18:13:11.510749 (+ 14054us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:11.510818 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0807 18:13:11.513453 (+ 2635us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:13:11.516996 (+ 3543us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:13:11.517010 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:13:11.518114 (+ 1104us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:13:11.519273 (+ 1159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:11.519309 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:11.521477 (+ 2168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:11.521485 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:11.524340 (+ 2855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:11.524360 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:11.525935 (+ 1575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:11.525943 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:11.526272 (+ 329us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:11.527162 (+ 890us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:13:11.527196 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:13:11.530707 (+ 3511us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:13:11.535783 (+ 5076us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:13:11.535793 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:13:11.535816 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:13:11.542335 (+ 6519us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:13:11.542932 (+ 597us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:13:11.542936 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:13:11.542939 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:13:11.543306 (+ 367us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:13:11.544143 (+ 837us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:13:11.544155 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:13:11.544812 (+ 657us) client_negotiation.cc:769] Sending connection context
0807 18:13:11.546871 (+ 2059us) client_negotiation.cc:240] Negotiation successful
0807 18:13:11.547267 (+ 396us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13678,"thread_start_us":19981,"threads_started":1}
I20260807 18:13:11.550626 29332 heartbeater.cc:347] Connected to a master server at 127.18.154.61:38131
I20260807 18:13:11.550999 28398 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:13:11.551046 29332 heartbeater.cc:464] Registering TS with master...
I20260807 18:13:11.551406 29330 heartbeater.cc:347] Connected to a master server at 127.18.154.60:36639
I20260807 18:13:11.551749 29330 heartbeater.cc:464] Registering TS with master...
I20260807 18:13:11.552371 29332 heartbeater.cc:511] Master 127.18.154.61:38131 requested a full tablet report, sending...
I20260807 18:13:11.552836 29330 heartbeater.cc:511] Master 127.18.154.60:36639 requested a full tablet report, sending...
I20260807 18:13:11.554560 28595 ts_manager.cc:194] Registered new tserver with Master: fde1306083f041638796f175e031fb30 (127.18.154.2:42923)
I20260807 18:13:11.555517 28874 ts_manager.cc:194] Registered new tserver with Master: fde1306083f041638796f175e031fb30 (127.18.154.2:42923)
I20260807 18:13:11.557978 29346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:11.517425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:34661 (local address 127.18.154.2:35055)
0807 18:13:11.524108 (+ 6683us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:11.524130 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 18:13:11.524249 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:13:11.535830 (+ 11581us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:13:11.535834 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:13:11.536537 (+ 703us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:13:11.537299 (+ 762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:11.537313 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:11.538920 (+ 1607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:11.538925 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:11.540834 (+ 1909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:11.540851 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:11.542451 (+ 1600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:11.542463 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:11.542696 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:11.543612 (+ 916us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:13:11.543634 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:13:11.549177 (+ 5543us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:13:11.554774 (+ 5597us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:13:11.554780 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:13:11.554784 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:13:11.555322 (+ 538us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:13:11.555800 (+ 478us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:13:11.555806 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:13:11.555809 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:13:11.556054 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:13:11.556800 (+ 746us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:13:11.556805 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:13:11.557036 (+ 231us) client_negotiation.cc:769] Sending connection context
0807 18:13:11.557191 (+ 155us) client_negotiation.cc:240] Negotiation successful
0807 18:13:11.557508 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6517,"thread_start_us":6888,"threads_started":1}
I20260807 18:13:11.558583 29351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:11.517602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.2:35055 (local address 127.18.154.62:34661)
0807 18:13:11.535166 (+ 17564us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:11.535176 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:11.535201 (+ 25us) server_negotiation.cc:407] Connection header received
0807 18:13:11.535275 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:11.535281 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:11.535383 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:11.535536 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:11.537483 (+ 1947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:11.538741 (+ 1258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:11.541047 (+ 2306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:11.541898 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:11.542026 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:11.549708 (+ 7682us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:13:11.549726 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:13:11.549731 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:13:11.549811 (+ 80us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:13:11.554548 (+ 4737us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:11.555479 (+ 931us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:11.555484 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:11.555490 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:11.555659 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:11.556202 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:11.556208 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:11.556212 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:11.556658 (+ 446us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:13:11.556810 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0807 18:13:11.557359 (+ 549us) server_negotiation.cc:299] Negotiation successful
0807 18:13:11.557787 (+ 428us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17411,"thread_start_us":18036,"threads_started":1}
I20260807 18:13:11.559338 29331 heartbeater.cc:347] Connected to a master server at 127.18.154.62:34661
I20260807 18:13:11.559489 29331 heartbeater.cc:464] Registering TS with master...
I20260807 18:13:11.559918 29331 heartbeater.cc:511] Master 127.18.154.62:34661 requested a full tablet report, sending...
I20260807 18:13:11.561022 27766 ts_manager.cc:194] Re-registered known tserver with Master: fde1306083f041638796f175e031fb30 (127.18.154.2:42923)
I20260807 18:13:11.562300 27766 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.18.154.2@KRBTEST.COM'} at 127.18.154.2:35055
I20260807 18:13:11.643424 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 28228
I20260807 18:13:11.667681 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.154.3:46263
--local_ip_for_outbound_sockets=127.18.154.3
--tserver_master_addrs=127.18.154.62:34661
--webserver_port=38257
--webserver_interface=127.18.154.3
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/krb5kdc/oryx_127.18.154.3.keytab
--principal=oryx/127.18.154.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.154.20:38893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.18.154.62:34661,127.18.154.61:38131
--tserver_master_addrs=127.18.154.62:34661,127.18.154.61:38131,127.18.154.60:36639 with env {KRB5CCNAME=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 18:13:11.871383 29133 heartbeater.cc:503] Master 127.18.154.62:34661 was elected leader, sending a full tablet report...
W20260807 18:13:12.025357 29353 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:13:12.025751 29353 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:13:12.025938 29353 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:13:12.033354 29353 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:13:12.033473 29353 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:13:12.033679 29353 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.154.3
I20260807 18:13:12.043862 29353 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.154.20:38893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/krb5kdc/oryx_127.18.154.3.keytab
--principal=oryx/127.18.154.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.154.3:46263
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.154.3
--webserver_port=38257
--tserver_master_addrs=127.18.154.62:34661,127.18.154.61:38131,127.18.154.60:36639
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.154.3
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:13:12.045346 29353 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:13:12.046777 29353 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:13:12.061267 29366 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:13:12.061408 29363 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:13:12.061381 29360 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:13:12.062480 29353 server_base.cc:1034] running on GCE node
I20260807 18:13:12.063104 29353 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:13:12.064114 29353 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:13:12.065322 29353 hybrid_clock.cc:648] HybridClock initialized: now 1786126392065276 us; error 38 us; skew 500 ppm
I20260807 18:13:12.065672 29353 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:13:12 dist-test-slave-gkw7 krb5kdc[27700](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126392, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.154.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:13:12.069223 29353 init.cc:377] Logged in from keytab as oryx/127.18.154.3@KRBTEST.COM (short username oryx)
I20260807 18:13:12.070868 29353 webserver.cc:460] Webserver started at http://127.18.154.3:38257/ using document root <none> and password file <none>
I20260807 18:13:12.071617 29353 fs_manager.cc:359] Metadata directory not provided
I20260807 18:13:12.071812 29353 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:13:12.077857 29353 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260807 18:13:12.080767 29378 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:13:12.082280 29353 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:13:12.082427 29353 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/ts-2/wal
uuid: "9b53136e1bd4469caee5c7f30bad8a38"
format_stamp: "Formatted at 2026-08-07 18:13:08 on dist-test-slave-gkw7"
server_key: "af387737d31f90b287d05e24f831e769"
server_key_iv: "eb906094aa4dfe59679b4a87edd7d701"
server_key_version: "encryptionkey@0"
I20260807 18:13:12.083482 29353 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:13:12.099854 29353 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:13:12.103816 29353 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:13:12.104132 29353 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:13:12.105242 29353 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:13:12.106675 29353 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:13:12.106799 29353 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:13:12.106936 29353 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:13:12.107044 29353 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:13:12.143836 29353 rpc_server.cc:307] RPC server started. Bound to: 127.18.154.3:46263
I20260807 18:13:12.143958 29543 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.154.3:46263 every 8 connection(s)
I20260807 18:13:12.145409 29353 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/ts-2/data/info.pb
I20260807 18:13:12.155488 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 29353
Aug 07 18:13:12 dist-test-slave-gkw7 krb5kdc[27700](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126392, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.154.3@KRBTEST.COM for oryx/127.18.154.62@KRBTEST.COM
Aug 07 18:13:12 dist-test-slave-gkw7 krb5kdc[27700](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126392, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.154.3@KRBTEST.COM for oryx/127.18.154.61@KRBTEST.COM
I20260807 18:13:12.176476 29556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:12.149441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:34661 (local address 127.18.154.3:45861)
0807 18:13:12.150904 (+ 1463us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:12.150984 (+ 80us) client_negotiation.cc:174] Beginning negotiation
0807 18:13:12.154679 (+ 3695us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:13:12.155722 (+ 1043us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:13:12.155730 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:13:12.156885 (+ 1155us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:13:12.158151 (+ 1266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:12.158180 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:12.159382 (+ 1202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:12.159388 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:12.160712 (+ 1324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:12.160725 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:12.161564 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:12.161573 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:12.161814 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:12.162593 (+ 779us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:13:12.162630 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:13:12.165792 (+ 3162us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:13:12.171211 (+ 5419us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:13:12.171222 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:13:12.171243 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:13:12.171981 (+ 738us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:13:12.172433 (+ 452us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:13:12.172438 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:13:12.172442 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:13:12.172677 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:13:12.173231 (+ 554us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:13:12.173255 (+ 24us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:13:12.173832 (+ 577us) client_negotiation.cc:769] Sending connection context
0807 18:13:12.174261 (+ 429us) client_negotiation.cc:240] Negotiation successful
0807 18:13:12.174843 (+ 582us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1025,"mutex_wait_us":151,"spinlock_wait_cycles":98176,"thread_start_us":397,"threads_started":1}
I20260807 18:13:12.177441 29553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:12.149260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.3:45861 (local address 127.18.154.62:34661)
0807 18:13:12.150477 (+ 1217us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:12.150486 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:12.151153 (+ 667us) server_negotiation.cc:407] Connection header received
0807 18:13:12.155095 (+ 3942us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:12.155105 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:12.155217 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:12.155304 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:12.158403 (+ 3099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:12.159209 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:12.160866 (+ 1657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:12.161293 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:12.161434 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:12.166030 (+ 4596us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:13:12.166036 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:13:12.166039 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:13:12.166074 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:13:12.170434 (+ 4360us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:12.172105 (+ 1671us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:12.172118 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:12.172121 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:12.172309 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:12.172801 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:12.172806 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:12.172811 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:12.173080 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:13:12.173306 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0807 18:13:12.174411 (+ 1105us) server_negotiation.cc:299] Negotiation successful
0807 18:13:12.174679 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1067,"thread_start_us":933,"threads_started":1}
I20260807 18:13:12.178987 29547 heartbeater.cc:347] Connected to a master server at 127.18.154.62:34661
I20260807 18:13:12.179486 29547 heartbeater.cc:464] Registering TS with master...
I20260807 18:13:12.180994 29547 heartbeater.cc:511] Master 127.18.154.62:34661 requested a full tablet report, sending...
I20260807 18:13:12.183308 27766 ts_manager.cc:194] Re-registered known tserver with Master: 9b53136e1bd4469caee5c7f30bad8a38 (127.18.154.3:46263)
I20260807 18:13:12.184477 27766 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.18.154.3@KRBTEST.COM'} at 127.18.154.3:45861
I20260807 18:13:12.184854 29554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:12.149481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.61:38131 (local address 127.18.154.3:36371)
0807 18:13:12.151078 (+ 1597us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:12.151099 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 18:13:12.154668 (+ 3569us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:13:12.155888 (+ 1220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:13:12.155895 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:13:12.156885 (+ 990us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:13:12.158151 (+ 1266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:12.158180 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:12.162917 (+ 4737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:12.162923 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:12.165173 (+ 2250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:12.165189 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:12.166447 (+ 1258us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:12.166456 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:12.166644 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:12.167242 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:13:12.167275 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:13:12.171026 (+ 3751us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:13:12.181175 (+ 10149us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:13:12.181180 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:13:12.181184 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:13:12.181744 (+ 560us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:13:12.182635 (+ 891us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:13:12.182641 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:13:12.182644 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:13:12.182905 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:13:12.183657 (+ 752us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:13:12.183662 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:13:12.183817 (+ 155us) client_negotiation.cc:769] Sending connection context
0807 18:13:12.183952 (+ 135us) client_negotiation.cc:240] Negotiation successful
0807 18:13:12.184331 (+ 379us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1464,"mutex_wait_us":167,"spinlock_wait_cycles":3840,"thread_start_us":2377,"threads_started":1}
Aug 07 18:13:12 dist-test-slave-gkw7 krb5kdc[27700](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126392, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.154.3@KRBTEST.COM for oryx/127.18.154.60@KRBTEST.COM
I20260807 18:13:12.186282 29551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:12.148888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.3:36371 (local address 127.18.154.61:38131)
0807 18:13:12.155493 (+ 6605us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:12.155500 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:12.155534 (+ 34us) server_negotiation.cc:407] Connection header received
0807 18:13:12.155606 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:12.155613 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:12.155708 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:12.155839 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:12.160379 (+ 4540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:12.162022 (+ 1643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:12.165326 (+ 3304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:12.166219 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:12.166416 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:12.171262 (+ 4846us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:13:12.171269 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:13:12.171274 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:13:12.171433 (+ 159us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:13:12.181025 (+ 9592us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:12.182251 (+ 1226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:12.182256 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:12.182258 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:12.182487 (+ 229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:12.183113 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:12.183118 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:12.183121 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:12.183510 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:13:12.185058 (+ 1548us) server_negotiation.cc:1035] Waiting for connection context
0807 18:13:12.185279 (+ 221us) server_negotiation.cc:299] Negotiation successful
0807 18:13:12.185674 (+ 395us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6421,"thread_start_us":6270,"threads_started":1}
I20260807 18:13:12.187433 29549 heartbeater.cc:347] Connected to a master server at 127.18.154.61:38131
I20260807 18:13:12.187680 29549 heartbeater.cc:464] Registering TS with master...
I20260807 18:13:12.188261 29549 heartbeater.cc:511] Master 127.18.154.61:38131 requested a full tablet report, sending...
I20260807 18:13:12.190078 28595 ts_manager.cc:194] Registered new tserver with Master: 9b53136e1bd4469caee5c7f30bad8a38 (127.18.154.3:46263)
I20260807 18:13:12.195752 29557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:12.151480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.60:36639 (local address 127.18.154.3:38169)
0807 18:13:12.152151 (+ 671us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:12.152175 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 18:13:12.155999 (+ 3824us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:13:12.168559 (+ 12560us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:13:12.168569 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:13:12.169821 (+ 1252us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:13:12.170424 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:12.170445 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:12.172816 (+ 2371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:12.172826 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:12.175388 (+ 2562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:12.175412 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:12.177288 (+ 1876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:12.177297 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:12.177657 (+ 360us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:12.178370 (+ 713us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:13:12.178405 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:13:12.186264 (+ 7859us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:13:12.190990 (+ 4726us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:13:12.190996 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:13:12.191000 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:13:12.191361 (+ 361us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:13:12.192551 (+ 1190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:13:12.192556 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:13:12.192560 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:13:12.192738 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:13:12.194271 (+ 1533us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:13:12.194279 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:13:12.194506 (+ 227us) client_negotiation.cc:769] Sending connection context
0807 18:13:12.194670 (+ 164us) client_negotiation.cc:240] Negotiation successful
0807 18:13:12.195123 (+ 453us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":503,"thread_start_us":813,"threads_started":1}
I20260807 18:13:12.198714 29559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:12.151910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.3:38169 (local address 127.18.154.60:36639)
0807 18:13:12.167901 (+ 15991us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:12.167913 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:12.167943 (+ 30us) server_negotiation.cc:407] Connection header received
0807 18:13:12.168028 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:12.168038 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:12.168156 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:12.170671 (+ 2515us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:12.170963 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:12.172562 (+ 1599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:12.175721 (+ 3159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:12.176958 (+ 1237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:12.178120 (+ 1162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:12.186523 (+ 8403us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:13:12.186532 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:13:12.186540 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:13:12.186594 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:13:12.190855 (+ 4261us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:12.191497 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:12.191502 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:12.191506 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:12.191695 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:12.192890 (+ 1195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:12.192896 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:12.192898 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:12.193979 (+ 1081us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:13:12.194175 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0807 18:13:12.196244 (+ 2069us) server_negotiation.cc:299] Negotiation successful
0807 18:13:12.196569 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15777,"thread_start_us":16390,"threads_started":1}
I20260807 18:13:12.199795 29546 heartbeater.cc:347] Connected to a master server at 127.18.154.60:36639
I20260807 18:13:12.199956 29546 heartbeater.cc:464] Registering TS with master...
I20260807 18:13:12.200488 29546 heartbeater.cc:511] Master 127.18.154.60:36639 requested a full tablet report, sending...
I20260807 18:13:12.202131 28874 ts_manager.cc:194] Registered new tserver with Master: 9b53136e1bd4469caee5c7f30bad8a38 (127.18.154.3:46263)
I20260807 18:13:12.213289 29553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:12.192033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52354 (local address 127.18.154.62:34661)
0807 18:13:12.192383 (+ 350us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:12.192392 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:12.192998 (+ 606us) server_negotiation.cc:407] Connection header received
0807 18:13:12.193258 (+ 260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:12.193267 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:12.193392 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:12.193518 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:12.196968 (+ 3450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:12.198326 (+ 1358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:12.200715 (+ 2389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:12.201346 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:12.201484 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:12.205727 (+ 4243us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:13:12.205733 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:13:12.205736 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:13:12.205770 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:13:12.209475 (+ 3705us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:12.210596 (+ 1121us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:12.210604 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:12.210609 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:12.210809 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:12.211346 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:12.211352 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:12.211356 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:12.211775 (+ 419us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:13:12.212151 (+ 376us) server_negotiation.cc:1035] Waiting for connection context
0807 18:13:12.212641 (+ 490us) server_negotiation.cc:299] Negotiation successful
0807 18:13:12.212901 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":189}
I20260807 18:13:12.220893 29551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:12.192748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52254 (local address 127.18.154.61:38131)
0807 18:13:12.198565 (+ 5817us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:12.198573 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:12.198593 (+ 20us) server_negotiation.cc:407] Connection header received
0807 18:13:12.198674 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:12.198680 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:12.198782 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:12.199777 (+ 995us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:12.204526 (+ 4749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:12.206048 (+ 1522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:12.207991 (+ 1943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:12.208763 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:12.208938 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:12.212937 (+ 3999us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:13:12.212943 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:13:12.212948 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:13:12.212986 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:13:12.217306 (+ 4320us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:12.218301 (+ 995us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:12.218309 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:12.218313 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:12.218585 (+ 272us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:12.218987 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:12.218993 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:12.218996 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:12.219396 (+ 400us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:13:12.219807 (+ 411us) server_negotiation.cc:1035] Waiting for connection context
0807 18:13:12.220164 (+ 357us) server_negotiation.cc:299] Negotiation successful
0807 18:13:12.220613 (+ 449us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5666}
I20260807 18:13:12.224658 29568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:12.193067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32900 (local address 127.18.154.60:36639)
0807 18:13:12.197310 (+ 4243us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:12.197321 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:12.206182 (+ 8861us) server_negotiation.cc:407] Connection header received
0807 18:13:12.206762 (+ 580us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:12.206769 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:12.206861 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:12.206970 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:12.209402 (+ 2432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:12.210680 (+ 1278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:12.212717 (+ 2037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:12.213449 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:12.213593 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:12.217219 (+ 3626us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:13:12.217225 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:13:12.217229 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:13:12.217261 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:13:12.221876 (+ 4615us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:12.222675 (+ 799us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:12.222679 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:12.222681 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:12.222834 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:12.223262 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:12.223265 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:12.223267 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:12.223539 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:13:12.223709 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0807 18:13:12.224158 (+ 449us) server_negotiation.cc:299] Negotiation successful
0807 18:13:12.224332 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4058,"thread_start_us":378,"threads_started":1}
I20260807 18:13:12.219458 27766 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52354:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 18:13:12.229390 27766 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:13:12.262982 29591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:12.250339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37866 (local address 127.18.154.2:42923)
0807 18:13:12.251025 (+ 686us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:12.251035 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:12.251189 (+ 154us) server_negotiation.cc:407] Connection header received
0807 18:13:12.251783 (+ 594us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:12.251794 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:12.252090 (+ 296us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:12.252371 (+ 281us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:13:12.253343 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:12.254620 (+ 1277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:12.259438 (+ 4818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:12.261327 (+ 1889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:12.261472 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:12.261806 (+ 334us) server_negotiation.cc:1035] Waiting for connection context
0807 18:13:12.262507 (+ 701us) server_negotiation.cc:299] Negotiation successful
0807 18:13:12.262688 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":426,"thread_start_us":262,"threads_started":1}
I20260807 18:13:12.263814 29589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:12.250041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.2:42923 (local address 127.0.0.1:37866)
0807 18:13:12.251067 (+ 1026us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:12.251085 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 18:13:12.251248 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:13:12.252584 (+ 1336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:13:12.252590 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:13:12.252619 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:13:12.253180 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:12.253197 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:12.255790 (+ 2593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:12.255796 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:12.259263 (+ 3467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:12.259279 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:12.261973 (+ 2694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:12.261980 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:12.262245 (+ 265us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:12.262290 (+ 45us) client_negotiation.cc:769] Sending connection context
0807 18:13:12.263281 (+ 991us) client_negotiation.cc:240] Negotiation successful
0807 18:13:12.263440 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":903,"mutex_wait_us":1,"thread_start_us":1344,"threads_started":1}
I20260807 18:13:12.266649 29592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:12.251567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.1:41283 (local address 127.0.0.1:49968)
0807 18:13:12.253721 (+ 2154us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:12.253741 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 18:13:12.253849 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:13:12.256836 (+ 2987us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:13:12.256843 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:13:12.256870 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:13:12.257786 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:12.257800 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:12.260793 (+ 2993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:12.260798 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:12.263487 (+ 2689us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:12.263500 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:12.265375 (+ 1875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:12.265382 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:12.265635 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:12.265681 (+ 46us) client_negotiation.cc:769] Sending connection context
0807 18:13:12.265781 (+ 100us) client_negotiation.cc:240] Negotiation successful
0807 18:13:12.265937 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2022,"mutex_wait_us":2,"spinlock_wait_cycles":491392,"thread_start_us":4681,"threads_started":1}
I20260807 18:13:12.266687 29593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:12.252497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35972 (local address 127.18.154.3:46263)
0807 18:13:12.253370 (+ 873us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:12.253383 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:12.256893 (+ 3510us) server_negotiation.cc:407] Connection header received
0807 18:13:12.257437 (+ 544us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:12.257448 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:12.257721 (+ 273us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:12.257922 (+ 201us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:13:12.258758 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:12.260544 (+ 1786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:12.262716 (+ 2172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:12.264297 (+ 1581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:12.264425 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:12.264632 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0807 18:13:12.266146 (+ 1514us) server_negotiation.cc:299] Negotiation successful
0807 18:13:12.266358 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":658,"thread_start_us":1189,"threads_started":1}
I20260807 18:13:12.266762 29599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:12.251778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.3:46263 (local address 127.0.0.1:35972)
0807 18:13:12.256705 (+ 4927us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:12.256783 (+ 78us) client_negotiation.cc:174] Beginning negotiation
0807 18:13:12.257186 (+ 403us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:13:12.258171 (+ 985us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:13:12.258174 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:13:12.258194 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:13:12.258565 (+ 371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:12.258573 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:12.260686 (+ 2113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:12.260702 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:12.262556 (+ 1854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:12.262569 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:12.265766 (+ 3197us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:12.265775 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:12.265965 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:12.265996 (+ 31us) client_negotiation.cc:769] Sending connection context
0807 18:13:12.266088 (+ 92us) client_negotiation.cc:240] Negotiation successful
0807 18:13:12.266228 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4770,"mutex_wait_us":52,"spinlock_wait_cycles":1920,"thread_start_us":3203,"threads_started":1}
I20260807 18:13:12.267346 29598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:12.253074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49968 (local address 127.18.154.1:41283)
0807 18:13:12.255175 (+ 2101us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:12.255183 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:12.255223 (+ 40us) server_negotiation.cc:407] Connection header received
0807 18:13:12.255325 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:12.255332 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:12.255595 (+ 263us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:12.255792 (+ 197us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:13:12.258041 (+ 2249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:12.259880 (+ 1839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:12.263658 (+ 3778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:12.265155 (+ 1497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:12.265299 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:12.265517 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0807 18:13:12.266925 (+ 1408us) server_negotiation.cc:299] Negotiation successful
0807 18:13:12.267099 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1919,"thread_start_us":1790,"threads_started":1}
I20260807 18:13:12.268183 29243 tablet_service.cc:1467] Processing CreateTablet for tablet 6ce3fdc5530e46fbb34448bdd4446026 (DEFAULT_TABLE table=test-table [id=1534b0667ffc4d00b7f2c7dc9928c60b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:13:12.269400 29243 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6ce3fdc5530e46fbb34448bdd4446026. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:13:12.273663 29034 tablet_service.cc:1467] Processing CreateTablet for tablet 6ce3fdc5530e46fbb34448bdd4446026 (DEFAULT_TABLE table=test-table [id=1534b0667ffc4d00b7f2c7dc9928c60b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:13:12.273819 29451 tablet_service.cc:1467] Processing CreateTablet for tablet 6ce3fdc5530e46fbb34448bdd4446026 (DEFAULT_TABLE table=test-table [id=1534b0667ffc4d00b7f2c7dc9928c60b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:13:12.275056 29034 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6ce3fdc5530e46fbb34448bdd4446026. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:13:12.275254 29451 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6ce3fdc5530e46fbb34448bdd4446026. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:13:12.289821 29602 tablet_bootstrap.cc:492] T 6ce3fdc5530e46fbb34448bdd4446026 P fde1306083f041638796f175e031fb30: Bootstrap starting.
I20260807 18:13:12.292964 29606 tablet_bootstrap.cc:492] T 6ce3fdc5530e46fbb34448bdd4446026 P 9b53136e1bd4469caee5c7f30bad8a38: Bootstrap starting.
I20260807 18:13:12.294174 29605 tablet_bootstrap.cc:492] T 6ce3fdc5530e46fbb34448bdd4446026 P 3d0c878c4dd74021bd3406b93a11c7c4: Bootstrap starting.
I20260807 18:13:12.295895 29602 tablet_bootstrap.cc:654] T 6ce3fdc5530e46fbb34448bdd4446026 P fde1306083f041638796f175e031fb30: Neither blocks nor log segments found. Creating new log.
I20260807 18:13:12.297045 29602 log.cc:824] T 6ce3fdc5530e46fbb34448bdd4446026 P fde1306083f041638796f175e031fb30: Log is configured to *not* fsync() on all Append() calls
I20260807 18:13:12.297106 29606 tablet_bootstrap.cc:654] T 6ce3fdc5530e46fbb34448bdd4446026 P 9b53136e1bd4469caee5c7f30bad8a38: Neither blocks nor log segments found. Creating new log.
I20260807 18:13:12.298251 29605 tablet_bootstrap.cc:654] T 6ce3fdc5530e46fbb34448bdd4446026 P 3d0c878c4dd74021bd3406b93a11c7c4: Neither blocks nor log segments found. Creating new log.
I20260807 18:13:12.298547 29606 log.cc:824] T 6ce3fdc5530e46fbb34448bdd4446026 P 9b53136e1bd4469caee5c7f30bad8a38: Log is configured to *not* fsync() on all Append() calls
I20260807 18:13:12.299750 29605 log.cc:824] T 6ce3fdc5530e46fbb34448bdd4446026 P 3d0c878c4dd74021bd3406b93a11c7c4: Log is configured to *not* fsync() on all Append() calls
I20260807 18:13:12.300716 29602 tablet_bootstrap.cc:492] T 6ce3fdc5530e46fbb34448bdd4446026 P fde1306083f041638796f175e031fb30: No bootstrap required, opened a new log
I20260807 18:13:12.301103 29602 ts_tablet_manager.cc:1397] T 6ce3fdc5530e46fbb34448bdd4446026 P fde1306083f041638796f175e031fb30: Time spent bootstrapping tablet: real 0.012s user 0.007s sys 0.000s
I20260807 18:13:12.303617 29605 tablet_bootstrap.cc:492] T 6ce3fdc5530e46fbb34448bdd4446026 P 3d0c878c4dd74021bd3406b93a11c7c4: No bootstrap required, opened a new log
I20260807 18:13:12.303681 29606 tablet_bootstrap.cc:492] T 6ce3fdc5530e46fbb34448bdd4446026 P 9b53136e1bd4469caee5c7f30bad8a38: No bootstrap required, opened a new log
I20260807 18:13:12.303921 29606 ts_tablet_manager.cc:1397] T 6ce3fdc5530e46fbb34448bdd4446026 P 9b53136e1bd4469caee5c7f30bad8a38: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.000s
I20260807 18:13:12.303923 29605 ts_tablet_manager.cc:1397] T 6ce3fdc5530e46fbb34448bdd4446026 P 3d0c878c4dd74021bd3406b93a11c7c4: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.004s
I20260807 18:13:12.309394 29602 raft_consensus.cc:348] T 6ce3fdc5530e46fbb34448bdd4446026 P fde1306083f041638796f175e031fb30 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fde1306083f041638796f175e031fb30" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 42923 } } peers { permanent_uuid: "9b53136e1bd4469caee5c7f30bad8a38" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 46263 } } peers { permanent_uuid: "3d0c878c4dd74021bd3406b93a11c7c4" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 41283 } }
I20260807 18:13:12.309912 29602 raft_consensus.cc:374] T 6ce3fdc5530e46fbb34448bdd4446026 P fde1306083f041638796f175e031fb30 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:13:12.310034 29602 raft_consensus.cc:729] T 6ce3fdc5530e46fbb34448bdd4446026 P fde1306083f041638796f175e031fb30 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fde1306083f041638796f175e031fb30, State: Initialized, Role: FOLLOWER
I20260807 18:13:12.310699 29602 consensus_queue.cc:260] T 6ce3fdc5530e46fbb34448bdd4446026 P fde1306083f041638796f175e031fb30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fde1306083f041638796f175e031fb30" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 42923 } } peers { permanent_uuid: "9b53136e1bd4469caee5c7f30bad8a38" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 46263 } } peers { permanent_uuid: "3d0c878c4dd74021bd3406b93a11c7c4" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 41283 } }
I20260807 18:13:12.312738 29602 ts_tablet_manager.cc:1428] T 6ce3fdc5530e46fbb34448bdd4446026 P fde1306083f041638796f175e031fb30: Time spent starting tablet: real 0.011s user 0.009s sys 0.004s
I20260807 18:13:12.313431 29331 heartbeater.cc:503] Master 127.18.154.62:34661 was elected leader, sending a full tablet report...
I20260807 18:13:12.316123 29606 raft_consensus.cc:348] T 6ce3fdc5530e46fbb34448bdd4446026 P 9b53136e1bd4469caee5c7f30bad8a38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fde1306083f041638796f175e031fb30" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 42923 } } peers { permanent_uuid: "9b53136e1bd4469caee5c7f30bad8a38" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 46263 } } peers { permanent_uuid: "3d0c878c4dd74021bd3406b93a11c7c4" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 41283 } }
I20260807 18:13:12.316237 29605 raft_consensus.cc:348] T 6ce3fdc5530e46fbb34448bdd4446026 P 3d0c878c4dd74021bd3406b93a11c7c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fde1306083f041638796f175e031fb30" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 42923 } } peers { permanent_uuid: "9b53136e1bd4469caee5c7f30bad8a38" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 46263 } } peers { permanent_uuid: "3d0c878c4dd74021bd3406b93a11c7c4" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 41283 } }
I20260807 18:13:12.316606 29606 raft_consensus.cc:374] T 6ce3fdc5530e46fbb34448bdd4446026 P 9b53136e1bd4469caee5c7f30bad8a38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:13:12.316619 29605 raft_consensus.cc:374] T 6ce3fdc5530e46fbb34448bdd4446026 P 3d0c878c4dd74021bd3406b93a11c7c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:13:12.316709 29606 raft_consensus.cc:729] T 6ce3fdc5530e46fbb34448bdd4446026 P 9b53136e1bd4469caee5c7f30bad8a38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b53136e1bd4469caee5c7f30bad8a38, State: Initialized, Role: FOLLOWER
I20260807 18:13:12.316747 29605 raft_consensus.cc:729] T 6ce3fdc5530e46fbb34448bdd4446026 P 3d0c878c4dd74021bd3406b93a11c7c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d0c878c4dd74021bd3406b93a11c7c4, State: Initialized, Role: FOLLOWER
I20260807 18:13:12.317461 29606 consensus_queue.cc:260] T 6ce3fdc5530e46fbb34448bdd4446026 P 9b53136e1bd4469caee5c7f30bad8a38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fde1306083f041638796f175e031fb30" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 42923 } } peers { permanent_uuid: "9b53136e1bd4469caee5c7f30bad8a38" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 46263 } } peers { permanent_uuid: "3d0c878c4dd74021bd3406b93a11c7c4" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 41283 } }
I20260807 18:13:12.317562 29605 consensus_queue.cc:260] T 6ce3fdc5530e46fbb34448bdd4446026 P 3d0c878c4dd74021bd3406b93a11c7c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fde1306083f041638796f175e031fb30" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 42923 } } peers { permanent_uuid: "9b53136e1bd4469caee5c7f30bad8a38" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 46263 } } peers { permanent_uuid: "3d0c878c4dd74021bd3406b93a11c7c4" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 41283 } }
I20260807 18:13:12.321998 29605 ts_tablet_manager.cc:1428] T 6ce3fdc5530e46fbb34448bdd4446026 P 3d0c878c4dd74021bd3406b93a11c7c4: Time spent starting tablet: real 0.018s user 0.009s sys 0.008s
I20260807 18:13:12.322204 29606 ts_tablet_manager.cc:1428] T 6ce3fdc5530e46fbb34448bdd4446026 P 9b53136e1bd4469caee5c7f30bad8a38: Time spent starting tablet: real 0.018s user 0.015s sys 0.005s
I20260807 18:13:12.323714 29547 heartbeater.cc:503] Master 127.18.154.62:34661 was elected leader, sending a full tablet report...
W20260807 18:13:12.400457 29550 tablet.cc:2369] T 6ce3fdc5530e46fbb34448bdd4446026 P 9b53136e1bd4469caee5c7f30bad8a38: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 18:13:12.505847 29336 tablet.cc:2369] T 6ce3fdc5530e46fbb34448bdd4446026 P fde1306083f041638796f175e031fb30: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 18:13:12.567183 29135 tablet.cc:2369] T 6ce3fdc5530e46fbb34448bdd4446026 P 3d0c878c4dd74021bd3406b93a11c7c4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:13:12.658816 29612 raft_consensus.cc:482] T 6ce3fdc5530e46fbb34448bdd4446026 P 3d0c878c4dd74021bd3406b93a11c7c4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:13:12.659345 29612 raft_consensus.cc:504] T 6ce3fdc5530e46fbb34448bdd4446026 P 3d0c878c4dd74021bd3406b93a11c7c4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fde1306083f041638796f175e031fb30" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 42923 } } peers { permanent_uuid: "9b53136e1bd4469caee5c7f30bad8a38" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 46263 } } peers { permanent_uuid: "3d0c878c4dd74021bd3406b93a11c7c4" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 41283 } }
I20260807 18:13:12.661156 29612 leader_election.cc:288] T 6ce3fdc5530e46fbb34448bdd4446026 P 3d0c878c4dd74021bd3406b93a11c7c4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fde1306083f041638796f175e031fb30 (127.18.154.2:42923), 9b53136e1bd4469caee5c7f30bad8a38 (127.18.154.3:46263)
I20260807 18:13:12.668861 29591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:12.661522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.1:59379 (local address 127.18.154.2:42923)
0807 18:13:12.661766 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:12.661770 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:12.662579 (+ 809us) server_negotiation.cc:407] Connection header received
0807 18:13:12.662915 (+ 336us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:12.662921 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:12.663006 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:12.663136 (+ 130us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:13:12.663837 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:12.664646 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:12.666914 (+ 2268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:12.667834 (+ 920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:12.668016 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:12.668103 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0807 18:13:12.668430 (+ 327us) server_negotiation.cc:299] Negotiation successful
0807 18:13:12.668558 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260807 18:13:12.669013 29615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:12.661329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.2:42923 (local address 127.18.154.1:59379)
0807 18:13:12.662447 (+ 1118us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:12.662481 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0807 18:13:12.662710 (+ 229us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:13:12.663152 (+ 442us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:13:12.663157 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:13:12.663185 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:13:12.663684 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:12.663695 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:12.664789 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:12.664794 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:12.666745 (+ 1951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:12.666759 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:12.668067 (+ 1308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:12.668074 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:12.668264 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:12.668290 (+ 26us) client_negotiation.cc:769] Sending connection context
0807 18:13:12.668405 (+ 115us) client_negotiation.cc:240] Negotiation successful
0807 18:13:12.668541 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":903,"thread_start_us":696,"threads_started":1}
I20260807 18:13:12.669154 29593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:12.662860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.1:38561 (local address 127.18.154.3:46263)
0807 18:13:12.663080 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:12.663087 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:12.663546 (+ 459us) server_negotiation.cc:407] Connection header received
0807 18:13:12.663752 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:12.663757 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:12.663825 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:12.663925 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:13:12.664464 (+ 539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:12.665231 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:12.667351 (+ 2120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:12.668160 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:12.668371 (+ 211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:12.668457 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0807 18:13:12.668788 (+ 331us) server_negotiation.cc:299] Negotiation successful
0807 18:13:12.668916 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 18:13:12.669737 29618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:12.662542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.3:46263 (local address 127.18.154.1:38561)
0807 18:13:12.663433 (+ 891us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:12.663448 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 18:13:12.663604 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:13:12.663944 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:13:12.663949 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:13:12.663973 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:13:12.664290 (+ 317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:12.664299 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:12.665385 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:12.665391 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:12.667181 (+ 1790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:12.667194 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:12.668467 (+ 1273us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:12.668477 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:12.668616 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:12.668641 (+ 25us) client_negotiation.cc:769] Sending connection context
0807 18:13:12.668750 (+ 109us) client_negotiation.cc:240] Negotiation successful
0807 18:13:12.668876 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":783,"thread_start_us":549,"threads_started":1}
I20260807 18:13:12.670493 29268 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ce3fdc5530e46fbb34448bdd4446026" candidate_uuid: "3d0c878c4dd74021bd3406b93a11c7c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fde1306083f041638796f175e031fb30" is_pre_election: true
I20260807 18:13:12.670853 29478 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ce3fdc5530e46fbb34448bdd4446026" candidate_uuid: "3d0c878c4dd74021bd3406b93a11c7c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b53136e1bd4469caee5c7f30bad8a38" is_pre_election: true
I20260807 18:13:12.671042 29268 raft_consensus.cc:2454] T 6ce3fdc5530e46fbb34448bdd4446026 P fde1306083f041638796f175e031fb30 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d0c878c4dd74021bd3406b93a11c7c4 in term 0.
I20260807 18:13:12.671466 29478 raft_consensus.cc:2454] T 6ce3fdc5530e46fbb34448bdd4446026 P 9b53136e1bd4469caee5c7f30bad8a38 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d0c878c4dd74021bd3406b93a11c7c4 in term 0.
I20260807 18:13:12.672165 28970 leader_election.cc:302] T 6ce3fdc5530e46fbb34448bdd4446026 P 3d0c878c4dd74021bd3406b93a11c7c4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d0c878c4dd74021bd3406b93a11c7c4, fde1306083f041638796f175e031fb30; no voters:
I20260807 18:13:12.672600 29612 raft_consensus.cc:2781] T 6ce3fdc5530e46fbb34448bdd4446026 P 3d0c878c4dd74021bd3406b93a11c7c4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:13:12.672817 29612 raft_consensus.cc:482] T 6ce3fdc5530e46fbb34448bdd4446026 P 3d0c878c4dd74021bd3406b93a11c7c4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:13:12.672950 29612 raft_consensus.cc:3037] T 6ce3fdc5530e46fbb34448bdd4446026 P 3d0c878c4dd74021bd3406b93a11c7c4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:13:12.675812 29612 raft_consensus.cc:504] T 6ce3fdc5530e46fbb34448bdd4446026 P 3d0c878c4dd74021bd3406b93a11c7c4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fde1306083f041638796f175e031fb30" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 42923 } } peers { permanent_uuid: "9b53136e1bd4469caee5c7f30bad8a38" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 46263 } } peers { permanent_uuid: "3d0c878c4dd74021bd3406b93a11c7c4" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 41283 } }
I20260807 18:13:12.676542 29612 leader_election.cc:288] T 6ce3fdc5530e46fbb34448bdd4446026 P 3d0c878c4dd74021bd3406b93a11c7c4 [CANDIDATE]: Term 1 election: Requested vote from peers fde1306083f041638796f175e031fb30 (127.18.154.2:42923), 9b53136e1bd4469caee5c7f30bad8a38 (127.18.154.3:46263)
I20260807 18:13:12.677301 29268 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ce3fdc5530e46fbb34448bdd4446026" candidate_uuid: "3d0c878c4dd74021bd3406b93a11c7c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fde1306083f041638796f175e031fb30"
I20260807 18:13:12.677599 29268 raft_consensus.cc:3037] T 6ce3fdc5530e46fbb34448bdd4446026 P fde1306083f041638796f175e031fb30 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:13:12.677740 29478 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ce3fdc5530e46fbb34448bdd4446026" candidate_uuid: "3d0c878c4dd74021bd3406b93a11c7c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b53136e1bd4469caee5c7f30bad8a38"
I20260807 18:13:12.677946 29478 raft_consensus.cc:3037] T 6ce3fdc5530e46fbb34448bdd4446026 P 9b53136e1bd4469caee5c7f30bad8a38 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:13:12.680374 29268 raft_consensus.cc:2454] T 6ce3fdc5530e46fbb34448bdd4446026 P fde1306083f041638796f175e031fb30 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d0c878c4dd74021bd3406b93a11c7c4 in term 1.
I20260807 18:13:12.680589 29478 raft_consensus.cc:2454] T 6ce3fdc5530e46fbb34448bdd4446026 P 9b53136e1bd4469caee5c7f30bad8a38 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d0c878c4dd74021bd3406b93a11c7c4 in term 1.
I20260807 18:13:12.680953 28970 leader_election.cc:302] T 6ce3fdc5530e46fbb34448bdd4446026 P 3d0c878c4dd74021bd3406b93a11c7c4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d0c878c4dd74021bd3406b93a11c7c4, fde1306083f041638796f175e031fb30; no voters:
I20260807 18:13:12.681393 29612 raft_consensus.cc:2781] T 6ce3fdc5530e46fbb34448bdd4446026 P 3d0c878c4dd74021bd3406b93a11c7c4 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:13:12.682866 29612 raft_consensus.cc:686] T 6ce3fdc5530e46fbb34448bdd4446026 P 3d0c878c4dd74021bd3406b93a11c7c4 [term 1 LEADER]: Becoming Leader. State: Replica: 3d0c878c4dd74021bd3406b93a11c7c4, State: Running, Role: LEADER
I20260807 18:13:12.683459 29612 consensus_queue.cc:237] T 6ce3fdc5530e46fbb34448bdd4446026 P 3d0c878c4dd74021bd3406b93a11c7c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fde1306083f041638796f175e031fb30" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 42923 } } peers { permanent_uuid: "9b53136e1bd4469caee5c7f30bad8a38" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 46263 } } peers { permanent_uuid: "3d0c878c4dd74021bd3406b93a11c7c4" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 41283 } }
I20260807 18:13:12.688843 27766 catalog_manager.cc:5495] T 6ce3fdc5530e46fbb34448bdd4446026 P 3d0c878c4dd74021bd3406b93a11c7c4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3d0c878c4dd74021bd3406b93a11c7c4 (127.18.154.1). New cstate: current_term: 1 leader_uuid: "3d0c878c4dd74021bd3406b93a11c7c4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fde1306083f041638796f175e031fb30" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 42923 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9b53136e1bd4469caee5c7f30bad8a38" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 46263 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d0c878c4dd74021bd3406b93a11c7c4" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 41283 } health_report { overall_health: HEALTHY } } }
I20260807 18:13:12.717723 29598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:12.711679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49976 (local address 127.18.154.1:41283)
0807 18:13:12.711973 (+ 294us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:12.711979 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:12.712000 (+ 21us) server_negotiation.cc:407] Connection header received
0807 18:13:12.712149 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:12.712154 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:12.712276 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:12.712400 (+ 124us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:13:12.712820 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:12.713870 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:12.715360 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:12.715925 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:12.716028 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:12.716561 (+ 533us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:13:12.717128 (+ 567us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:13:12.717262 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0807 18:13:12.717307 (+ 45us) server_negotiation.cc:299] Negotiation successful
0807 18:13:12.717376 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 18:13:12.726886 29268 raft_consensus.cc:1261] T 6ce3fdc5530e46fbb34448bdd4446026 P fde1306083f041638796f175e031fb30 [term 1 FOLLOWER]: Refusing update from remote peer 3d0c878c4dd74021bd3406b93a11c7c4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:13:12.727061 29478 raft_consensus.cc:1261] T 6ce3fdc5530e46fbb34448bdd4446026 P 9b53136e1bd4469caee5c7f30bad8a38 [term 1 FOLLOWER]: Refusing update from remote peer 3d0c878c4dd74021bd3406b93a11c7c4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:13:12.728113 29621 consensus_queue.cc:1035] T 6ce3fdc5530e46fbb34448bdd4446026 P 3d0c878c4dd74021bd3406b93a11c7c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fde1306083f041638796f175e031fb30" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 42923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:13:12.728546 29612 consensus_queue.cc:1035] T 6ce3fdc5530e46fbb34448bdd4446026 P 3d0c878c4dd74021bd3406b93a11c7c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9b53136e1bd4469caee5c7f30bad8a38" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 46263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:13:12.743575 29626 mvcc.cc:205] Tried to move back new op lower bound from 7315973704598380544 to 7315973704435539968. Current Snapshot: MvccSnapshot[applied={T|T < 7315973704598380544}]
I20260807 18:13:12.759970 29628 mvcc.cc:205] Tried to move back new op lower bound from 7315973704598380544 to 7315973704435539968. Current Snapshot: MvccSnapshot[applied={T|T < 7315973704598380544}]
I20260807 18:13:12.772121 27766 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52354:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:13:12.772586 27766 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52354:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:13:12.779181 27766 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cc97b2cc29884038b29e0e18de9a6d18: Sending DeleteTablet for 3 replicas of tablet 6ce3fdc5530e46fbb34448bdd4446026
I20260807 18:13:12.780576 29243 tablet_service.cc:1514] Processing DeleteTablet for tablet 6ce3fdc5530e46fbb34448bdd4446026 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:13:12 UTC) from {username='oryx', principal='oryx/127.18.154.62@KRBTEST.COM'} at 127.0.0.1:37866
I20260807 18:13:12.781060 29451 tablet_service.cc:1514] Processing DeleteTablet for tablet 6ce3fdc5530e46fbb34448bdd4446026 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:13:12 UTC) from {username='oryx', principal='oryx/127.18.154.62@KRBTEST.COM'} at 127.0.0.1:35972
I20260807 18:13:12.781602 29034 tablet_service.cc:1514] Processing DeleteTablet for tablet 6ce3fdc5530e46fbb34448bdd4446026 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:13:12 UTC) from {username='oryx', principal='oryx/127.18.154.62@KRBTEST.COM'} at 127.0.0.1:49968
I20260807 18:13:12.782213 29641 tablet_replica.cc:330] T 6ce3fdc5530e46fbb34448bdd4446026 P 9b53136e1bd4469caee5c7f30bad8a38: stopping tablet replica
I20260807 18:13:12.782802 29641 raft_consensus.cc:2229] T 6ce3fdc5530e46fbb34448bdd4446026 P 9b53136e1bd4469caee5c7f30bad8a38 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:13:12.783633 29641 raft_consensus.cc:2258] T 6ce3fdc5530e46fbb34448bdd4446026 P 9b53136e1bd4469caee5c7f30bad8a38 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:13:12.785908 29643 tablet_replica.cc:330] T 6ce3fdc5530e46fbb34448bdd4446026 P 3d0c878c4dd74021bd3406b93a11c7c4: stopping tablet replica
I20260807 18:13:12.786460 29641 ts_tablet_manager.cc:1905] T 6ce3fdc5530e46fbb34448bdd4446026 P 9b53136e1bd4469caee5c7f30bad8a38: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:13:12.786525 29643 raft_consensus.cc:2229] T 6ce3fdc5530e46fbb34448bdd4446026 P 3d0c878c4dd74021bd3406b93a11c7c4 [term 1 LEADER]: Raft consensus shutting down.
I20260807 18:13:12.787242 29643 raft_consensus.cc:2258] T 6ce3fdc5530e46fbb34448bdd4446026 P 3d0c878c4dd74021bd3406b93a11c7c4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:13:12.788947 29643 ts_tablet_manager.cc:1905] T 6ce3fdc5530e46fbb34448bdd4446026 P 3d0c878c4dd74021bd3406b93a11c7c4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:13:12.795147 29643 ts_tablet_manager.cc:1918] T 6ce3fdc5530e46fbb34448bdd4446026 P 3d0c878c4dd74021bd3406b93a11c7c4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:13:12.795499 29643 log.cc:1196] T 6ce3fdc5530e46fbb34448bdd4446026 P 3d0c878c4dd74021bd3406b93a11c7c4: Deleting WAL directory at /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/ts-0/wal/wals/6ce3fdc5530e46fbb34448bdd4446026
I20260807 18:13:12.796307 29643 ts_tablet_manager.cc:1939] T 6ce3fdc5530e46fbb34448bdd4446026 P 3d0c878c4dd74021bd3406b93a11c7c4: Deleting consensus metadata
I20260807 18:13:12.798951 27744 catalog_manager.cc:4841] TS 3d0c878c4dd74021bd3406b93a11c7c4 (127.18.154.1:41283): tablet 6ce3fdc5530e46fbb34448bdd4446026 (table test-table [id=1534b0667ffc4d00b7f2c7dc9928c60b]) successfully deleted
I20260807 18:13:12.800047 29641 ts_tablet_manager.cc:1918] T 6ce3fdc5530e46fbb34448bdd4446026 P 9b53136e1bd4469caee5c7f30bad8a38: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:13:12.800271 29641 log.cc:1196] T 6ce3fdc5530e46fbb34448bdd4446026 P 9b53136e1bd4469caee5c7f30bad8a38: Deleting WAL directory at /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/ts-2/wal/wals/6ce3fdc5530e46fbb34448bdd4446026
I20260807 18:13:12.801282 29641 ts_tablet_manager.cc:1939] T 6ce3fdc5530e46fbb34448bdd4446026 P 9b53136e1bd4469caee5c7f30bad8a38: Deleting consensus metadata
I20260807 18:13:12.803828 27745 catalog_manager.cc:4841] TS 9b53136e1bd4469caee5c7f30bad8a38 (127.18.154.3:46263): tablet 6ce3fdc5530e46fbb34448bdd4446026 (table test-table [id=1534b0667ffc4d00b7f2c7dc9928c60b]) successfully deleted
I20260807 18:13:12.804309 29640 tablet_replica.cc:330] T 6ce3fdc5530e46fbb34448bdd4446026 P fde1306083f041638796f175e031fb30: stopping tablet replica
I20260807 18:13:12.804762 29640 raft_consensus.cc:2229] T 6ce3fdc5530e46fbb34448bdd4446026 P fde1306083f041638796f175e031fb30 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:13:12.805353 29640 raft_consensus.cc:2258] T 6ce3fdc5530e46fbb34448bdd4446026 P fde1306083f041638796f175e031fb30 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:13:12.807487 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 27717
I20260807 18:13:12.808179 29640 ts_tablet_manager.cc:1905] T 6ce3fdc5530e46fbb34448bdd4446026 P fde1306083f041638796f175e031fb30: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:13:12.808655 27806 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:13:12.824986 29640 ts_tablet_manager.cc:1918] T 6ce3fdc5530e46fbb34448bdd4446026 P fde1306083f041638796f175e031fb30: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:13:12.825193 29640 log.cc:1196] T 6ce3fdc5530e46fbb34448bdd4446026 P fde1306083f041638796f175e031fb30: Deleting WAL directory at /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/ts-1/wal/wals/6ce3fdc5530e46fbb34448bdd4446026
I20260807 18:13:12.825963 29640 ts_tablet_manager.cc:1939] T 6ce3fdc5530e46fbb34448bdd4446026 P fde1306083f041638796f175e031fb30: Deleting consensus metadata
I20260807 18:13:12.843417 29645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:12.785652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52256 (local address 127.18.154.61:38131)
0807 18:13:12.786327 (+ 675us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:12.786334 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:12.806319 (+ 19985us) server_negotiation.cc:407] Connection header received
0807 18:13:12.806949 (+ 630us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:12.806956 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:12.807056 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:12.807173 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:12.811744 (+ 4571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:12.812964 (+ 1220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:12.815016 (+ 2052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:12.815964 (+ 948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:12.816067 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:12.831858 (+ 15791us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:13:12.831867 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:13:12.831872 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:13:12.831924 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:13:12.836570 (+ 4646us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:12.837573 (+ 1003us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:12.837579 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:12.837584 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:12.837738 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:12.838175 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:12.838181 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:12.838186 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:12.838563 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:13:12.838865 (+ 302us) server_negotiation.cc:1035] Waiting for connection context
0807 18:13:12.842719 (+ 3854us) server_negotiation.cc:299] Negotiation successful
0807 18:13:12.843019 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":555,"thread_start_us":1042,"threads_started":1}
I20260807 18:13:12.856290 29648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:12.806694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32906 (local address 127.18.154.60:36639)
0807 18:13:12.809018 (+ 2324us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:12.809028 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:12.840143 (+ 31115us) server_negotiation.cc:407] Connection header received
0807 18:13:12.840383 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:12.840389 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:12.840487 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:12.840573 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:12.842611 (+ 2038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:12.843883 (+ 1272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:12.846013 (+ 2130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:12.846505 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:12.846675 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:12.849811 (+ 3136us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:13:12.849816 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:13:12.849819 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:13:12.849883 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:13:12.853202 (+ 3319us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:12.854039 (+ 837us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:12.854046 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:12.854052 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:12.854201 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:12.854603 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:12.854608 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:12.854611 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:12.854932 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:13:12.855225 (+ 293us) server_negotiation.cc:1035] Waiting for connection context
0807 18:13:12.855527 (+ 302us) server_negotiation.cc:299] Negotiation successful
0807 18:13:12.855823 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2132,"thread_start_us":1866,"threads_started":1}
I20260807 18:13:12.999794 27745 catalog_manager.cc:4841] TS fde1306083f041638796f175e031fb30 (127.18.154.2:42923): tablet 6ce3fdc5530e46fbb34448bdd4446026 (table test-table [id=1534b0667ffc4d00b7f2c7dc9928c60b]) successfully deleted
I20260807 18:13:13.002100 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 27717
I20260807 18:13:13.054342 29648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:13.036840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32920 (local address 127.18.154.60:36639)
0807 18:13:13.037076 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:13.037085 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:13.038415 (+ 1330us) server_negotiation.cc:407] Connection header received
0807 18:13:13.038644 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:13.038649 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:13.038715 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:13.038824 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:13.040667 (+ 1843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:13.041806 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:13.043660 (+ 1854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:13.044120 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:13.044271 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:13.047607 (+ 3336us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:13:13.047612 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:13:13.047616 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:13:13.047653 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:13:13.051075 (+ 3422us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:13.051855 (+ 780us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:13.051861 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:13.051865 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:13.052044 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:13.052495 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:13.052500 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:13.052504 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:13.052815 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:13:13.053090 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0807 18:13:13.053672 (+ 582us) server_negotiation.cc:299] Negotiation successful
0807 18:13:13.054086 (+ 414us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260807 18:13:13.055577 29645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:13.035663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52270 (local address 127.18.154.61:38131)
0807 18:13:13.035929 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:13.035939 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:13.036949 (+ 1010us) server_negotiation.cc:407] Connection header received
0807 18:13:13.037252 (+ 303us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:13.037259 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:13.037346 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:13.037425 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:13.040220 (+ 2795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:13.041345 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:13.043495 (+ 2150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:13.044184 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:13.044315 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:13.048075 (+ 3760us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:13:13.048082 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:13:13.048087 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:13:13.048145 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:13:13.052094 (+ 3949us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:13.053010 (+ 916us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:13.053016 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:13.053019 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:13.053186 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:13.053686 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:13.053693 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:13.053696 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:13.054045 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:13:13.054325 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0807 18:13:13.054930 (+ 605us) server_negotiation.cc:299] Negotiation successful
0807 18:13:13.055184 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 18:13:13.749245 29658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:13.748447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:34661 (local address 127.18.154.2:58437)
0807 18:13:13.748950 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:13.749071 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.62:34661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":400,"thread_start_us":271,"threads_started":1}
W20260807 18:13:13.749889 29331 heartbeater.cc:644] Failed to heartbeat to 127.18.154.62:34661 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.18.154.62:34661: connect: Connection refused (error 111)
I20260807 18:13:13.761387 29659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:13.760679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:34661 (local address 127.18.154.1:49897)
0807 18:13:13.761101 (+ 422us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:13.761224 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.62:34661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":319,"thread_start_us":210,"threads_started":1}
W20260807 18:13:13.761858 29133 heartbeater.cc:644] Failed to heartbeat to 127.18.154.62:34661 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.18.154.62:34661: connect: Connection refused (error 111)
I20260807 18:13:13.767035 29660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:13.766028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:34661 (local address 127.18.154.3:46581)
0807 18:13:13.766621 (+ 593us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:13.766792 (+ 171us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.62:34661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":482,"thread_start_us":374,"threads_started":1}
W20260807 18:13:13.767479 29547 heartbeater.cc:644] Failed to heartbeat to 127.18.154.62:34661 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.18.154.62:34661: connect: Connection refused (error 111)
I20260807 18:13:14.279919 29661 raft_consensus.cc:482] T 00000000000000000000000000000000 P a6983037814748628555659bcf04b165 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cc97b2cc29884038b29e0e18de9a6d18)
I20260807 18:13:14.280248 29662 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3d6d83d5ddcb4df3bcba443888204a34 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cc97b2cc29884038b29e0e18de9a6d18)
I20260807 18:13:14.280272 29661 raft_consensus.cc:504] T 00000000000000000000000000000000 P a6983037814748628555659bcf04b165 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "cc97b2cc29884038b29e0e18de9a6d18" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 34661 } } peers { permanent_uuid: "a6983037814748628555659bcf04b165" member_type: VOTER last_known_addr { host: "127.18.154.61" port: 38131 } attrs { promote: false } } peers { permanent_uuid: "3d6d83d5ddcb4df3bcba443888204a34" member_type: VOTER last_known_addr { host: "127.18.154.60" port: 36639 } attrs { promote: false } }
I20260807 18:13:14.280537 29662 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3d6d83d5ddcb4df3bcba443888204a34 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "cc97b2cc29884038b29e0e18de9a6d18" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 34661 } } peers { permanent_uuid: "a6983037814748628555659bcf04b165" member_type: VOTER last_known_addr { host: "127.18.154.61" port: 38131 } attrs { promote: false } } peers { permanent_uuid: "3d6d83d5ddcb4df3bcba443888204a34" member_type: VOTER last_known_addr { host: "127.18.154.60" port: 36639 } attrs { promote: false } }
I20260807 18:13:14.281926 29661 leader_election.cc:288] T 00000000000000000000000000000000 P a6983037814748628555659bcf04b165 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cc97b2cc29884038b29e0e18de9a6d18 (127.18.154.62:34661), 3d6d83d5ddcb4df3bcba443888204a34 (127.18.154.60:36639)
I20260807 18:13:14.282004 29662 leader_election.cc:288] T 00000000000000000000000000000000 P 3d6d83d5ddcb4df3bcba443888204a34 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cc97b2cc29884038b29e0e18de9a6d18 (127.18.154.62:34661), a6983037814748628555659bcf04b165 (127.18.154.61:38131)
I20260807 18:13:14.283226 29664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:14.282104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:34661 (local address 127.0.0.1:52754)
0807 18:13:14.282682 (+ 578us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:14.282908 (+ 226us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.62:34661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":401,"thread_start_us":240,"threads_started":1}
I20260807 18:13:14.285001 29666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:14.282170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:34661 (local address 127.0.0.1:52762)
0807 18:13:14.284434 (+ 2264us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:14.284645 (+ 211us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.62:34661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2113,"threads_started":1}
W20260807 18:13:14.290024 28577 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.154.62:34661: connect: Connection refused (error 111)
I20260807 18:13:14.295176 29664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:14.282708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.60:36639 (local address 127.0.0.1:32934)
0807 18:13:14.283631 (+ 923us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:14.283654 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 18:13:14.283825 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:13:14.286567 (+ 2742us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:13:14.286577 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:13:14.286646 (+ 69us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:13:14.287030 (+ 384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:14.287045 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:14.289864 (+ 2819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:14.289873 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:14.292243 (+ 2370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:14.292261 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:14.294042 (+ 1781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:14.294049 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:14.294348 (+ 299us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:14.294408 (+ 60us) client_negotiation.cc:769] Sending connection context
0807 18:13:14.294497 (+ 89us) client_negotiation.cc:240] Negotiation successful
0807 18:13:14.294722 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":786,"thread_start_us":6885,"threads_started":1}
I20260807 18:13:14.296535 29673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:14.283148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32934 (local address 127.18.154.60:36639)
0807 18:13:14.286142 (+ 2994us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:14.286151 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:14.286174 (+ 23us) server_negotiation.cc:407] Connection header received
0807 18:13:14.286249 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:14.286257 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:14.286355 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:14.287258 (+ 903us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:13:14.287593 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:14.289228 (+ 1635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:14.292497 (+ 3269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:14.293770 (+ 1273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:14.295724 (+ 1954us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:14.295863 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0807 18:13:14.295919 (+ 56us) server_negotiation.cc:299] Negotiation successful
0807 18:13:14.296046 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2822,"thread_start_us":2674,"threads_started":1}
W20260807 18:13:14.297503 28854 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.154.62:34661: connect: Connection refused (error 111)
I20260807 18:13:14.298172 28886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a6983037814748628555659bcf04b165" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "3d6d83d5ddcb4df3bcba443888204a34" is_pre_election: true
I20260807 18:13:14.298568 28886 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 3d6d83d5ddcb4df3bcba443888204a34 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a6983037814748628555659bcf04b165 in term 1.
I20260807 18:13:14.299358 29664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:14.298923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:34661 (local address 127.0.0.1:52780)
0807 18:13:14.299117 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:14.299208 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.62:34661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260807 18:13:14.299690 28577 leader_election.cc:334] T 00000000000000000000000000000000 P a6983037814748628555659bcf04b165 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer cc97b2cc29884038b29e0e18de9a6d18 (127.18.154.62:34661): Network error: Client connection negotiation failed: client connection to 127.18.154.62:34661: connect: Connection refused (error 111)
I20260807 18:13:14.300081 28577 leader_election.cc:302] T 00000000000000000000000000000000 P a6983037814748628555659bcf04b165 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d6d83d5ddcb4df3bcba443888204a34, a6983037814748628555659bcf04b165; no voters: cc97b2cc29884038b29e0e18de9a6d18
I20260807 18:13:14.300524 29665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:14.299876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:34661 (local address 127.0.0.1:52792)
0807 18:13:14.300058 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:14.300272 (+ 214us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.62:34661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260807 18:13:14.300526 29661 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a6983037814748628555659bcf04b165 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 18:13:14.300661 29661 raft_consensus.cc:482] T 00000000000000000000000000000000 P a6983037814748628555659bcf04b165 [term 1 FOLLOWER]: Starting leader election (detected failure of leader cc97b2cc29884038b29e0e18de9a6d18)
W20260807 18:13:14.300722 28854 leader_election.cc:334] T 00000000000000000000000000000000 P 3d6d83d5ddcb4df3bcba443888204a34 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer cc97b2cc29884038b29e0e18de9a6d18 (127.18.154.62:34661): Network error: Client connection negotiation failed: client connection to 127.18.154.62:34661: connect: Connection refused (error 111)
I20260807 18:13:14.300788 29661 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a6983037814748628555659bcf04b165 [term 1 FOLLOWER]: Advancing to term 2
I20260807 18:13:14.302268 29666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:14.282408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.61:38131 (local address 127.0.0.1:52284)
0807 18:13:14.285340 (+ 2932us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:14.285362 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 18:13:14.285515 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:13:14.295269 (+ 9754us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:13:14.295275 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:13:14.295303 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:13:14.295562 (+ 259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:14.295574 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:14.297815 (+ 2241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:14.297821 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:14.300012 (+ 2191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:14.300027 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:14.301476 (+ 1449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:14.301484 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:14.301688 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:14.301722 (+ 34us) client_negotiation.cc:769] Sending connection context
0807 18:13:14.301840 (+ 118us) client_negotiation.cc:240] Negotiation successful
0807 18:13:14.301980 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2808,"thread_start_us":1696,"threads_started":1}
I20260807 18:13:14.302299 29671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:14.282727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52284 (local address 127.18.154.61:38131)
0807 18:13:14.294484 (+ 11757us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:14.294493 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:14.294520 (+ 27us) server_negotiation.cc:407] Connection header received
0807 18:13:14.294597 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:14.294605 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:14.294704 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:14.295722 (+ 1018us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:13:14.296043 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:14.297619 (+ 1576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:14.300183 (+ 2564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:14.301287 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:14.301425 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:14.301589 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0807 18:13:14.301863 (+ 274us) server_negotiation.cc:299] Negotiation successful
0807 18:13:14.301967 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11608,"thread_start_us":12094,"threads_started":1}
I20260807 18:13:14.303260 28611 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3d6d83d5ddcb4df3bcba443888204a34" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "a6983037814748628555659bcf04b165" is_pre_election: true
I20260807 18:13:14.303795 29661 raft_consensus.cc:504] T 00000000000000000000000000000000 P a6983037814748628555659bcf04b165 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "cc97b2cc29884038b29e0e18de9a6d18" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 34661 } } peers { permanent_uuid: "a6983037814748628555659bcf04b165" member_type: VOTER last_known_addr { host: "127.18.154.61" port: 38131 } attrs { promote: false } } peers { permanent_uuid: "3d6d83d5ddcb4df3bcba443888204a34" member_type: VOTER last_known_addr { host: "127.18.154.60" port: 36639 } attrs { promote: false } }
I20260807 18:13:14.304322 28611 raft_consensus.cc:2379] T 00000000000000000000000000000000 P a6983037814748628555659bcf04b165 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d6d83d5ddcb4df3bcba443888204a34 in current term 2: Already voted for candidate a6983037814748628555659bcf04b165 in this term.
I20260807 18:13:14.304577 29661 leader_election.cc:288] T 00000000000000000000000000000000 P a6983037814748628555659bcf04b165 [CANDIDATE]: Term 2 election: Requested vote from peers cc97b2cc29884038b29e0e18de9a6d18 (127.18.154.62:34661), 3d6d83d5ddcb4df3bcba443888204a34 (127.18.154.60:36639)
I20260807 18:13:14.304980 29664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:14.304646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:34661 (local address 127.0.0.1:52800)
0807 18:13:14.304764 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:14.304850 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.62:34661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260807 18:13:14.305183 28886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a6983037814748628555659bcf04b165" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "3d6d83d5ddcb4df3bcba443888204a34"
I20260807 18:13:14.305255 28856 leader_election.cc:302] T 00000000000000000000000000000000 P 3d6d83d5ddcb4df3bcba443888204a34 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d6d83d5ddcb4df3bcba443888204a34; no voters: a6983037814748628555659bcf04b165, cc97b2cc29884038b29e0e18de9a6d18
I20260807 18:13:14.305469 28886 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3d6d83d5ddcb4df3bcba443888204a34 [term 1 FOLLOWER]: Advancing to term 2
I20260807 18:13:14.306031 29664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:14.305710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:34661 (local address 127.0.0.1:52808)
0807 18:13:14.305854 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:14.305926 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.62:34661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260807 18:13:14.306228 28577 leader_election.cc:334] T 00000000000000000000000000000000 P a6983037814748628555659bcf04b165 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer cc97b2cc29884038b29e0e18de9a6d18 (127.18.154.62:34661): Network error: Client connection negotiation failed: client connection to 127.18.154.62:34661: connect: Connection refused (error 111)
I20260807 18:13:14.307762 28886 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 3d6d83d5ddcb4df3bcba443888204a34 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a6983037814748628555659bcf04b165 in term 2.
I20260807 18:13:14.307941 29662 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 3d6d83d5ddcb4df3bcba443888204a34 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 18:13:14.308175 28577 leader_election.cc:302] T 00000000000000000000000000000000 P a6983037814748628555659bcf04b165 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d6d83d5ddcb4df3bcba443888204a34, a6983037814748628555659bcf04b165; no voters: cc97b2cc29884038b29e0e18de9a6d18
I20260807 18:13:14.308458 29661 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a6983037814748628555659bcf04b165 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 18:13:14.308852 29661 raft_consensus.cc:686] T 00000000000000000000000000000000 P a6983037814748628555659bcf04b165 [term 2 LEADER]: Becoming Leader. State: Replica: a6983037814748628555659bcf04b165, State: Running, Role: LEADER
I20260807 18:13:14.309234 29661 consensus_queue.cc:237] T 00000000000000000000000000000000 P a6983037814748628555659bcf04b165 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "cc97b2cc29884038b29e0e18de9a6d18" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 34661 } } peers { permanent_uuid: "a6983037814748628555659bcf04b165" member_type: VOTER last_known_addr { host: "127.18.154.61" port: 38131 } attrs { promote: false } } peers { permanent_uuid: "3d6d83d5ddcb4df3bcba443888204a34" member_type: VOTER last_known_addr { host: "127.18.154.60" port: 36639 } attrs { promote: false } }
I20260807 18:13:14.310971 29676 sys_catalog.cc:455] T 00000000000000000000000000000000 P a6983037814748628555659bcf04b165 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a6983037814748628555659bcf04b165. Latest consensus state: current_term: 2 leader_uuid: "a6983037814748628555659bcf04b165" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "cc97b2cc29884038b29e0e18de9a6d18" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 34661 } } peers { permanent_uuid: "a6983037814748628555659bcf04b165" member_type: VOTER last_known_addr { host: "127.18.154.61" port: 38131 } attrs { promote: false } } peers { permanent_uuid: "3d6d83d5ddcb4df3bcba443888204a34" member_type: VOTER last_known_addr { host: "127.18.154.60" port: 36639 } attrs { promote: false } } }
I20260807 18:13:14.311450 29676 sys_catalog.cc:458] T 00000000000000000000000000000000 P a6983037814748628555659bcf04b165 [sys.catalog]: This master's current role is: LEADER
I20260807 18:13:14.312111 29679 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:13:14.316290 29679 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:13:14.317118 29679 catalog_manager.cc:1259] Loaded cluster ID: 6f80460f0d5e43d69126321a9a079ffa
I20260807 18:13:14.317217 29679 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:13:14.318454 29679 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:13:14.319278 29679 catalog_manager.cc:5879] T 00000000000000000000000000000000 P a6983037814748628555659bcf04b165: Loaded TSK: 0
I20260807 18:13:14.320204 29679 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:13:14.353170 28595 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52270:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 18:13:14.360504 28595 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:13:14.364840 29664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:14.364394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:34661 (local address 127.0.0.1:52826)
0807 18:13:14.364532 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:14.364666 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.62:34661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260807 18:13:14.365243 28886 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 3d6d83d5ddcb4df3bcba443888204a34 [term 2 FOLLOWER]: Refusing update from remote peer a6983037814748628555659bcf04b165: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260807 18:13:14.365947 29664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:14.365488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:34661 (local address 127.0.0.1:52828)
0807 18:13:14.365696 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:14.365778 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.62:34661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260807 18:13:14.366052 29676 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a6983037814748628555659bcf04b165 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d6d83d5ddcb4df3bcba443888204a34" member_type: VOTER last_known_addr { host: "127.18.154.60" port: 36639 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260807 18:13:14.366362 28577 consensus_peers.cc:487] T 00000000000000000000000000000000 P a6983037814748628555659bcf04b165 -> Peer cc97b2cc29884038b29e0e18de9a6d18 (127.18.154.62:34661): Couldn't send request to peer cc97b2cc29884038b29e0e18de9a6d18. Status: Network error: Client connection negotiation failed: client connection to 127.18.154.62:34661: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260807 18:13:14.368829 29662 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d6d83d5ddcb4df3bcba443888204a34 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a6983037814748628555659bcf04b165. Latest consensus state: current_term: 2 leader_uuid: "a6983037814748628555659bcf04b165" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "cc97b2cc29884038b29e0e18de9a6d18" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 34661 } } peers { permanent_uuid: "a6983037814748628555659bcf04b165" member_type: VOTER last_known_addr { host: "127.18.154.61" port: 38131 } attrs { promote: false } } peers { permanent_uuid: "3d6d83d5ddcb4df3bcba443888204a34" member_type: VOTER last_known_addr { host: "127.18.154.60" port: 36639 } attrs { promote: false } } }
I20260807 18:13:14.369066 29662 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d6d83d5ddcb4df3bcba443888204a34 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 18:13:14.371325 29661 sys_catalog.cc:455] T 00000000000000000000000000000000 P a6983037814748628555659bcf04b165 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "a6983037814748628555659bcf04b165" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "cc97b2cc29884038b29e0e18de9a6d18" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 34661 } } peers { permanent_uuid: "a6983037814748628555659bcf04b165" member_type: VOTER last_known_addr { host: "127.18.154.61" port: 38131 } attrs { promote: false } } peers { permanent_uuid: "3d6d83d5ddcb4df3bcba443888204a34" member_type: VOTER last_known_addr { host: "127.18.154.60" port: 36639 } attrs { promote: false } } }
I20260807 18:13:14.371627 29661 sys_catalog.cc:458] T 00000000000000000000000000000000 P a6983037814748628555659bcf04b165 [sys.catalog]: This master's current role is: LEADER
I20260807 18:13:14.372068 29662 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d6d83d5ddcb4df3bcba443888204a34 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "a6983037814748628555659bcf04b165" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "cc97b2cc29884038b29e0e18de9a6d18" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 34661 } } peers { permanent_uuid: "a6983037814748628555659bcf04b165" member_type: VOTER last_known_addr { host: "127.18.154.61" port: 38131 } attrs { promote: false } } peers { permanent_uuid: "3d6d83d5ddcb4df3bcba443888204a34" member_type: VOTER last_known_addr { host: "127.18.154.60" port: 36639 } attrs { promote: false } } }
I20260807 18:13:14.372350 29662 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d6d83d5ddcb4df3bcba443888204a34 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 18:13:14.387818 29669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:14.380473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.3:46263 (local address 127.0.0.1:35980)
0807 18:13:14.380624 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:14.380648 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 18:13:14.380774 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:13:14.382144 (+ 1370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:13:14.382151 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:13:14.382181 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:13:14.382621 (+ 440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:14.382631 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:14.383814 (+ 1183us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:14.383820 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:14.385755 (+ 1935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:14.385771 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:14.387112 (+ 1341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:14.387118 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:14.387288 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:14.387314 (+ 26us) client_negotiation.cc:769] Sending connection context
0807 18:13:14.387370 (+ 56us) client_negotiation.cc:240] Negotiation successful
0807 18:13:14.387515 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34,"mutex_wait_us":87}
I20260807 18:13:14.388684 29664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:14.379486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.2:42923 (local address 127.0.0.1:37878)
0807 18:13:14.379675 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:14.379695 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 18:13:14.379791 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:13:14.380510 (+ 719us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:13:14.380518 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:13:14.380546 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:13:14.380775 (+ 229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:14.380794 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:14.382934 (+ 2140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:14.382943 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:14.385755 (+ 2812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:14.385771 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:14.387912 (+ 2141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:14.387917 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:14.388082 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:14.388116 (+ 34us) client_negotiation.cc:769] Sending connection context
0807 18:13:14.388230 (+ 114us) client_negotiation.cc:240] Negotiation successful
0807 18:13:14.388369 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60,"mutex_wait_us":70}
I20260807 18:13:14.389415 29686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:14.380611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35980 (local address 127.18.154.3:46263)
0807 18:13:14.381085 (+ 474us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:14.381092 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:14.381110 (+ 18us) server_negotiation.cc:407] Connection header received
0807 18:13:14.381214 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:14.381219 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:14.381298 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:14.381635 (+ 337us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:13:14.382797 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:14.383675 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:14.385952 (+ 2277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:14.386871 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:14.388633 (+ 1762us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:14.388956 (+ 323us) server_negotiation.cc:1035] Waiting for connection context
0807 18:13:14.389008 (+ 52us) server_negotiation.cc:299] Negotiation successful
0807 18:13:14.389143 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":361,"thread_start_us":255,"threads_started":1}
I20260807 18:13:14.389796 29685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:14.379643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37878 (local address 127.18.154.2:42923)
0807 18:13:14.380150 (+ 507us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:14.380157 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:14.380178 (+ 21us) server_negotiation.cc:407] Connection header received
0807 18:13:14.380255 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:14.380260 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:14.380339 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:14.380965 (+ 626us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:13:14.381185 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:14.382721 (+ 1536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:14.385932 (+ 3211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:14.387672 (+ 1740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:14.387890 (+ 218us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:14.388043 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0807 18:13:14.388890 (+ 847us) server_negotiation.cc:299] Negotiation successful
0807 18:13:14.389478 (+ 588us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":373,"thread_start_us":275,"threads_started":1}
I20260807 18:13:14.390332 29451 tablet_service.cc:1467] Processing CreateTablet for tablet bd55b293540a4feb8daa06ea4220b5db (DEFAULT_TABLE table=test-table [id=82e685f489af40f3b3448215cbab10da]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:13:14.390472 29243 tablet_service.cc:1467] Processing CreateTablet for tablet bd55b293540a4feb8daa06ea4220b5db (DEFAULT_TABLE table=test-table [id=82e685f489af40f3b3448215cbab10da]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:13:14.390815 29451 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bd55b293540a4feb8daa06ea4220b5db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:13:14.390987 29243 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bd55b293540a4feb8daa06ea4220b5db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:13:14.395372 29669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:14.381431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.1:41283 (local address 127.0.0.1:49984)
0807 18:13:14.388312 (+ 6881us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:14.388329 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 18:13:14.388448 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:13:14.389221 (+ 773us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:13:14.389229 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:13:14.389267 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:13:14.389492 (+ 225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:14.389506 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:14.391056 (+ 1550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:14.391062 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:14.392815 (+ 1753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:14.392840 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:14.394543 (+ 1703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:14.394551 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:14.394773 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:14.394806 (+ 33us) client_negotiation.cc:769] Sending connection context
0807 18:13:14.394885 (+ 79us) client_negotiation.cc:240] Negotiation successful
0807 18:13:14.395034 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6762,"threads_started":1}
I20260807 18:13:14.396209 29690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:14.381912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49984 (local address 127.18.154.1:41283)
0807 18:13:14.382402 (+ 490us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:14.382411 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:14.388414 (+ 6003us) server_negotiation.cc:407] Connection header received
0807 18:13:14.388597 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:14.388606 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:14.388711 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:14.389683 (+ 972us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:13:14.389881 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:14.390900 (+ 1019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:14.393030 (+ 2130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:14.394355 (+ 1325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:14.395578 (+ 1223us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:14.395684 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0807 18:13:14.395735 (+ 51us) server_negotiation.cc:299] Negotiation successful
0807 18:13:14.395866 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":362,"thread_start_us":860,"threads_started":1}
I20260807 18:13:14.398149 29692 tablet_bootstrap.cc:492] T bd55b293540a4feb8daa06ea4220b5db P fde1306083f041638796f175e031fb30: Bootstrap starting.
I20260807 18:13:14.399046 29034 tablet_service.cc:1467] Processing CreateTablet for tablet bd55b293540a4feb8daa06ea4220b5db (DEFAULT_TABLE table=test-table [id=82e685f489af40f3b3448215cbab10da]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:13:14.399308 29694 tablet_bootstrap.cc:492] T bd55b293540a4feb8daa06ea4220b5db P 9b53136e1bd4469caee5c7f30bad8a38: Bootstrap starting.
I20260807 18:13:14.399538 29034 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bd55b293540a4feb8daa06ea4220b5db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:13:14.400740 29694 tablet_bootstrap.cc:654] T bd55b293540a4feb8daa06ea4220b5db P 9b53136e1bd4469caee5c7f30bad8a38: Neither blocks nor log segments found. Creating new log.
I20260807 18:13:14.402894 29694 tablet_bootstrap.cc:492] T bd55b293540a4feb8daa06ea4220b5db P 9b53136e1bd4469caee5c7f30bad8a38: No bootstrap required, opened a new log
I20260807 18:13:14.403043 29694 ts_tablet_manager.cc:1397] T bd55b293540a4feb8daa06ea4220b5db P 9b53136e1bd4469caee5c7f30bad8a38: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20260807 18:13:14.403342 29692 tablet_bootstrap.cc:654] T bd55b293540a4feb8daa06ea4220b5db P fde1306083f041638796f175e031fb30: Neither blocks nor log segments found. Creating new log.
I20260807 18:13:14.403945 29694 raft_consensus.cc:348] T bd55b293540a4feb8daa06ea4220b5db P 9b53136e1bd4469caee5c7f30bad8a38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fde1306083f041638796f175e031fb30" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 42923 } } peers { permanent_uuid: "9b53136e1bd4469caee5c7f30bad8a38" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 46263 } } peers { permanent_uuid: "3d0c878c4dd74021bd3406b93a11c7c4" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 41283 } }
I20260807 18:13:14.404222 29694 raft_consensus.cc:374] T bd55b293540a4feb8daa06ea4220b5db P 9b53136e1bd4469caee5c7f30bad8a38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:13:14.404313 29694 raft_consensus.cc:729] T bd55b293540a4feb8daa06ea4220b5db P 9b53136e1bd4469caee5c7f30bad8a38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b53136e1bd4469caee5c7f30bad8a38, State: Initialized, Role: FOLLOWER
I20260807 18:13:14.404539 29694 consensus_queue.cc:260] T bd55b293540a4feb8daa06ea4220b5db P 9b53136e1bd4469caee5c7f30bad8a38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fde1306083f041638796f175e031fb30" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 42923 } } peers { permanent_uuid: "9b53136e1bd4469caee5c7f30bad8a38" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 46263 } } peers { permanent_uuid: "3d0c878c4dd74021bd3406b93a11c7c4" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 41283 } }
I20260807 18:13:14.406605 29698 tablet_bootstrap.cc:492] T bd55b293540a4feb8daa06ea4220b5db P 3d0c878c4dd74021bd3406b93a11c7c4: Bootstrap starting.
I20260807 18:13:14.409185 29698 tablet_bootstrap.cc:654] T bd55b293540a4feb8daa06ea4220b5db P 3d0c878c4dd74021bd3406b93a11c7c4: Neither blocks nor log segments found. Creating new log.
I20260807 18:13:14.410494 29694 ts_tablet_manager.cc:1428] T bd55b293540a4feb8daa06ea4220b5db P 9b53136e1bd4469caee5c7f30bad8a38: Time spent starting tablet: real 0.007s user 0.003s sys 0.005s
I20260807 18:13:14.417670 29692 tablet_bootstrap.cc:492] T bd55b293540a4feb8daa06ea4220b5db P fde1306083f041638796f175e031fb30: No bootstrap required, opened a new log
I20260807 18:13:14.417836 29692 ts_tablet_manager.cc:1397] T bd55b293540a4feb8daa06ea4220b5db P fde1306083f041638796f175e031fb30: Time spent bootstrapping tablet: real 0.020s user 0.010s sys 0.000s
I20260807 18:13:14.417831 29549 heartbeater.cc:503] Master 127.18.154.61:38131 was elected leader, sending a full tablet report...
I20260807 18:13:14.418730 29692 raft_consensus.cc:348] T bd55b293540a4feb8daa06ea4220b5db P fde1306083f041638796f175e031fb30 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fde1306083f041638796f175e031fb30" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 42923 } } peers { permanent_uuid: "9b53136e1bd4469caee5c7f30bad8a38" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 46263 } } peers { permanent_uuid: "3d0c878c4dd74021bd3406b93a11c7c4" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 41283 } }
I20260807 18:13:14.419045 29692 raft_consensus.cc:374] T bd55b293540a4feb8daa06ea4220b5db P fde1306083f041638796f175e031fb30 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:13:14.419170 29692 raft_consensus.cc:729] T bd55b293540a4feb8daa06ea4220b5db P fde1306083f041638796f175e031fb30 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fde1306083f041638796f175e031fb30, State: Initialized, Role: FOLLOWER
I20260807 18:13:14.419507 29692 consensus_queue.cc:260] T bd55b293540a4feb8daa06ea4220b5db P fde1306083f041638796f175e031fb30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fde1306083f041638796f175e031fb30" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 42923 } } peers { permanent_uuid: "9b53136e1bd4469caee5c7f30bad8a38" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 46263 } } peers { permanent_uuid: "3d0c878c4dd74021bd3406b93a11c7c4" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 41283 } }
I20260807 18:13:14.421820 29702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:14.411301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:34661 (local address 127.18.154.3:35233)
0807 18:13:14.421254 (+ 9953us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:14.421477 (+ 223us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.62:34661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":9791,"thread_start_us":9646,"threads_started":1}
I20260807 18:13:14.423027 29702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:14.422529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:34661 (local address 127.18.154.3:40229)
0807 18:13:14.422722 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:14.422836 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.62:34661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260807 18:13:14.423576 29698 tablet_bootstrap.cc:492] T bd55b293540a4feb8daa06ea4220b5db P 3d0c878c4dd74021bd3406b93a11c7c4: No bootstrap required, opened a new log
I20260807 18:13:14.423748 29698 ts_tablet_manager.cc:1397] T bd55b293540a4feb8daa06ea4220b5db P 3d0c878c4dd74021bd3406b93a11c7c4: Time spent bootstrapping tablet: real 0.017s user 0.007s sys 0.007s
I20260807 18:13:14.424427 29698 raft_consensus.cc:348] T bd55b293540a4feb8daa06ea4220b5db P 3d0c878c4dd74021bd3406b93a11c7c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fde1306083f041638796f175e031fb30" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 42923 } } peers { permanent_uuid: "9b53136e1bd4469caee5c7f30bad8a38" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 46263 } } peers { permanent_uuid: "3d0c878c4dd74021bd3406b93a11c7c4" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 41283 } }
I20260807 18:13:14.424634 29698 raft_consensus.cc:374] T bd55b293540a4feb8daa06ea4220b5db P 3d0c878c4dd74021bd3406b93a11c7c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:13:14.424731 29698 raft_consensus.cc:729] T bd55b293540a4feb8daa06ea4220b5db P 3d0c878c4dd74021bd3406b93a11c7c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d0c878c4dd74021bd3406b93a11c7c4, State: Initialized, Role: FOLLOWER
I20260807 18:13:14.424991 29698 consensus_queue.cc:260] T bd55b293540a4feb8daa06ea4220b5db P 3d0c878c4dd74021bd3406b93a11c7c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fde1306083f041638796f175e031fb30" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 42923 } } peers { permanent_uuid: "9b53136e1bd4469caee5c7f30bad8a38" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 46263 } } peers { permanent_uuid: "3d0c878c4dd74021bd3406b93a11c7c4" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 41283 } }
I20260807 18:13:14.425958 29698 ts_tablet_manager.cc:1428] T bd55b293540a4feb8daa06ea4220b5db P 3d0c878c4dd74021bd3406b93a11c7c4: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260807 18:13:14.426718 29692 ts_tablet_manager.cc:1428] T bd55b293540a4feb8daa06ea4220b5db P fde1306083f041638796f175e031fb30: Time spent starting tablet: real 0.009s user 0.003s sys 0.004s
I20260807 18:13:14.428596 29134 heartbeater.cc:503] Master 127.18.154.61:38131 was elected leader, sending a full tablet report...
I20260807 18:13:14.429564 29706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:14.426990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:34661 (local address 127.18.154.2:45787)
0807 18:13:14.427689 (+ 699us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:14.427971 (+ 282us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.62:34661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":495,"thread_start_us":308,"threads_started":1}
I20260807 18:13:14.429854 29332 heartbeater.cc:503] Master 127.18.154.61:38131 was elected leader, sending a full tablet report...
I20260807 18:13:14.432515 29707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:14.427079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:34661 (local address 127.18.154.1:47907)
0807 18:13:14.432050 (+ 4971us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:14.432222 (+ 172us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.62:34661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":4812,"thread_start_us":4663,"threads_started":1}
I20260807 18:13:14.433329 29706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:14.432067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:34661 (local address 127.18.154.2:51277)
0807 18:13:14.433143 (+ 1076us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:14.433212 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.62:34661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":476,"thread_start_us":885,"threads_started":1}
I20260807 18:13:14.433646 29707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:14.433172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:34661 (local address 127.18.154.1:38303)
0807 18:13:14.433334 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:14.433432 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.62:34661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260807 18:13:14.506798 29705 raft_consensus.cc:482] T bd55b293540a4feb8daa06ea4220b5db P 3d0c878c4dd74021bd3406b93a11c7c4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:13:14.507084 29705 raft_consensus.cc:504] T bd55b293540a4feb8daa06ea4220b5db P 3d0c878c4dd74021bd3406b93a11c7c4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fde1306083f041638796f175e031fb30" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 42923 } } peers { permanent_uuid: "9b53136e1bd4469caee5c7f30bad8a38" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 46263 } } peers { permanent_uuid: "3d0c878c4dd74021bd3406b93a11c7c4" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 41283 } }
I20260807 18:13:14.507918 29705 leader_election.cc:288] T bd55b293540a4feb8daa06ea4220b5db P 3d0c878c4dd74021bd3406b93a11c7c4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fde1306083f041638796f175e031fb30 (127.18.154.2:42923), 9b53136e1bd4469caee5c7f30bad8a38 (127.18.154.3:46263)
I20260807 18:13:14.508524 29267 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd55b293540a4feb8daa06ea4220b5db" candidate_uuid: "3d0c878c4dd74021bd3406b93a11c7c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fde1306083f041638796f175e031fb30" is_pre_election: true
I20260807 18:13:14.508714 29478 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd55b293540a4feb8daa06ea4220b5db" candidate_uuid: "3d0c878c4dd74021bd3406b93a11c7c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b53136e1bd4469caee5c7f30bad8a38" is_pre_election: true
I20260807 18:13:14.508859 29267 raft_consensus.cc:2454] T bd55b293540a4feb8daa06ea4220b5db P fde1306083f041638796f175e031fb30 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d0c878c4dd74021bd3406b93a11c7c4 in term 0.
I20260807 18:13:14.509027 29478 raft_consensus.cc:2454] T bd55b293540a4feb8daa06ea4220b5db P 9b53136e1bd4469caee5c7f30bad8a38 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d0c878c4dd74021bd3406b93a11c7c4 in term 0.
I20260807 18:13:14.509480 28970 leader_election.cc:302] T bd55b293540a4feb8daa06ea4220b5db P 3d0c878c4dd74021bd3406b93a11c7c4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d0c878c4dd74021bd3406b93a11c7c4, fde1306083f041638796f175e031fb30; no voters:
I20260807 18:13:14.509825 29705 raft_consensus.cc:2781] T bd55b293540a4feb8daa06ea4220b5db P 3d0c878c4dd74021bd3406b93a11c7c4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:13:14.509963 29705 raft_consensus.cc:482] T bd55b293540a4feb8daa06ea4220b5db P 3d0c878c4dd74021bd3406b93a11c7c4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:13:14.510120 29705 raft_consensus.cc:3037] T bd55b293540a4feb8daa06ea4220b5db P 3d0c878c4dd74021bd3406b93a11c7c4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:13:14.512774 29705 raft_consensus.cc:504] T bd55b293540a4feb8daa06ea4220b5db P 3d0c878c4dd74021bd3406b93a11c7c4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fde1306083f041638796f175e031fb30" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 42923 } } peers { permanent_uuid: "9b53136e1bd4469caee5c7f30bad8a38" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 46263 } } peers { permanent_uuid: "3d0c878c4dd74021bd3406b93a11c7c4" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 41283 } }
I20260807 18:13:14.513425 29705 leader_election.cc:288] T bd55b293540a4feb8daa06ea4220b5db P 3d0c878c4dd74021bd3406b93a11c7c4 [CANDIDATE]: Term 1 election: Requested vote from peers fde1306083f041638796f175e031fb30 (127.18.154.2:42923), 9b53136e1bd4469caee5c7f30bad8a38 (127.18.154.3:46263)
I20260807 18:13:14.513958 29267 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd55b293540a4feb8daa06ea4220b5db" candidate_uuid: "3d0c878c4dd74021bd3406b93a11c7c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fde1306083f041638796f175e031fb30"
I20260807 18:13:14.514194 29267 raft_consensus.cc:3037] T bd55b293540a4feb8daa06ea4220b5db P fde1306083f041638796f175e031fb30 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:13:14.514137 29478 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd55b293540a4feb8daa06ea4220b5db" candidate_uuid: "3d0c878c4dd74021bd3406b93a11c7c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b53136e1bd4469caee5c7f30bad8a38"
I20260807 18:13:14.514367 29478 raft_consensus.cc:3037] T bd55b293540a4feb8daa06ea4220b5db P 9b53136e1bd4469caee5c7f30bad8a38 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:13:14.516741 29267 raft_consensus.cc:2454] T bd55b293540a4feb8daa06ea4220b5db P fde1306083f041638796f175e031fb30 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d0c878c4dd74021bd3406b93a11c7c4 in term 1.
I20260807 18:13:14.517177 28970 leader_election.cc:302] T bd55b293540a4feb8daa06ea4220b5db P 3d0c878c4dd74021bd3406b93a11c7c4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d0c878c4dd74021bd3406b93a11c7c4, fde1306083f041638796f175e031fb30; no voters:
I20260807 18:13:14.517388 29478 raft_consensus.cc:2454] T bd55b293540a4feb8daa06ea4220b5db P 9b53136e1bd4469caee5c7f30bad8a38 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d0c878c4dd74021bd3406b93a11c7c4 in term 1.
I20260807 18:13:14.517498 29705 raft_consensus.cc:2781] T bd55b293540a4feb8daa06ea4220b5db P 3d0c878c4dd74021bd3406b93a11c7c4 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:13:14.518425 29705 raft_consensus.cc:686] T bd55b293540a4feb8daa06ea4220b5db P 3d0c878c4dd74021bd3406b93a11c7c4 [term 1 LEADER]: Becoming Leader. State: Replica: 3d0c878c4dd74021bd3406b93a11c7c4, State: Running, Role: LEADER
I20260807 18:13:14.519049 29705 consensus_queue.cc:237] T bd55b293540a4feb8daa06ea4220b5db P 3d0c878c4dd74021bd3406b93a11c7c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fde1306083f041638796f175e031fb30" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 42923 } } peers { permanent_uuid: "9b53136e1bd4469caee5c7f30bad8a38" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 46263 } } peers { permanent_uuid: "3d0c878c4dd74021bd3406b93a11c7c4" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 41283 } }
I20260807 18:13:14.519929 29707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:14.519445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:34661 (local address 127.18.154.1:44541)
0807 18:13:14.519622 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:14.519739 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.62:34661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260807 18:13:14.522320 28595 catalog_manager.cc:5495] T bd55b293540a4feb8daa06ea4220b5db P 3d0c878c4dd74021bd3406b93a11c7c4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3d0c878c4dd74021bd3406b93a11c7c4 (127.18.154.1). New cstate: current_term: 1 leader_uuid: "3d0c878c4dd74021bd3406b93a11c7c4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fde1306083f041638796f175e031fb30" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 42923 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9b53136e1bd4469caee5c7f30bad8a38" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 46263 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d0c878c4dd74021bd3406b93a11c7c4" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 41283 } health_report { overall_health: HEALTHY } } }
I20260807 18:13:14.563949 29690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:14.556425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49998 (local address 127.18.154.1:41283)
0807 18:13:14.556762 (+ 337us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:14.556771 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:14.556998 (+ 227us) server_negotiation.cc:407] Connection header received
0807 18:13:14.557188 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:14.557193 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:14.557289 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:14.557407 (+ 118us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:13:14.557922 (+ 515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:14.559234 (+ 1312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:14.561448 (+ 2214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:14.562294 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:14.562440 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:14.563025 (+ 585us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:13:14.563221 (+ 196us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:13:14.563333 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0807 18:13:14.563507 (+ 174us) server_negotiation.cc:299] Negotiation successful
0807 18:13:14.563621 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147}
I20260807 18:13:14.570502 29267 raft_consensus.cc:1261] T bd55b293540a4feb8daa06ea4220b5db P fde1306083f041638796f175e031fb30 [term 1 FOLLOWER]: Refusing update from remote peer 3d0c878c4dd74021bd3406b93a11c7c4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:13:14.570540 29478 raft_consensus.cc:1261] T bd55b293540a4feb8daa06ea4220b5db P 9b53136e1bd4469caee5c7f30bad8a38 [term 1 FOLLOWER]: Refusing update from remote peer 3d0c878c4dd74021bd3406b93a11c7c4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:13:14.571211 29711 consensus_queue.cc:1035] T bd55b293540a4feb8daa06ea4220b5db P 3d0c878c4dd74021bd3406b93a11c7c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fde1306083f041638796f175e031fb30" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 42923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:13:14.571671 29705 consensus_queue.cc:1035] T bd55b293540a4feb8daa06ea4220b5db P 3d0c878c4dd74021bd3406b93a11c7c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9b53136e1bd4469caee5c7f30bad8a38" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 46263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:13:14.574717 29706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:14.574142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:34661 (local address 127.18.154.2:42265)
0807 18:13:14.574380 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:14.574477 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.62:34661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94}
I20260807 18:13:14.575451 29702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:14.574969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:34661 (local address 127.18.154.3:44077)
0807 18:13:14.575149 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:14.575270 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.62:34661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260807 18:13:14.578646 29716 mvcc.cc:205] Tried to move back new op lower bound from 7315973712154046464 to 7315973711953100800. Current Snapshot: MvccSnapshot[applied={T|T < 7315973712154046464}]
I20260807 18:13:14.580482 29706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:14.579880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:34661 (local address 127.18.154.2:53971)
0807 18:13:14.580150 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:14.580274 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.62:34661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131}
I20260807 18:13:14.585402 29702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:14.584647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:34661 (local address 127.18.154.3:57773)
0807 18:13:14.585119 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:14.585211 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.62:34661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":352}
I20260807 18:13:14.595331 29707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:14.594313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:34661 (local address 127.18.154.1:47045)
0807 18:13:14.595044 (+ 731us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:14.595141 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.62:34661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":618}
I20260807 18:13:14.596846 29707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:14.596195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:34661 (local address 127.18.154.1:38807)
0807 18:13:14.596578 (+ 383us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:14.596674 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.62:34661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":275}
I20260807 18:13:14.598459 28595 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52270:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:13:14.598856 28595 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52270:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:13:14.605458 28595 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a6983037814748628555659bcf04b165: Sending DeleteTablet for 3 replicas of tablet bd55b293540a4feb8daa06ea4220b5db
I20260807 18:13:14.606895 29034 tablet_service.cc:1514] Processing DeleteTablet for tablet bd55b293540a4feb8daa06ea4220b5db with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:13:14 UTC) from {username='oryx', principal='oryx/127.18.154.61@KRBTEST.COM'} at 127.0.0.1:49984
I20260807 18:13:14.606967 29451 tablet_service.cc:1514] Processing DeleteTablet for tablet bd55b293540a4feb8daa06ea4220b5db with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:13:14 UTC) from {username='oryx', principal='oryx/127.18.154.61@KRBTEST.COM'} at 127.0.0.1:35980
I20260807 18:13:14.607599 29243 tablet_service.cc:1514] Processing DeleteTablet for tablet bd55b293540a4feb8daa06ea4220b5db with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:13:14 UTC) from {username='oryx', principal='oryx/127.18.154.61@KRBTEST.COM'} at 127.0.0.1:37878
I20260807 18:13:14.609200 29734 tablet_replica.cc:330] T bd55b293540a4feb8daa06ea4220b5db P 9b53136e1bd4469caee5c7f30bad8a38: stopping tablet replica
I20260807 18:13:14.609462 29734 raft_consensus.cc:2229] T bd55b293540a4feb8daa06ea4220b5db P 9b53136e1bd4469caee5c7f30bad8a38 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:13:14.609866 29734 raft_consensus.cc:2258] T bd55b293540a4feb8daa06ea4220b5db P 9b53136e1bd4469caee5c7f30bad8a38 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:13:14.610517 29731 tablet_replica.cc:330] T bd55b293540a4feb8daa06ea4220b5db P 3d0c878c4dd74021bd3406b93a11c7c4: stopping tablet replica
I20260807 18:13:14.610769 29731 raft_consensus.cc:2229] T bd55b293540a4feb8daa06ea4220b5db P 3d0c878c4dd74021bd3406b93a11c7c4 [term 1 LEADER]: Raft consensus shutting down.
I20260807 18:13:14.611332 29731 raft_consensus.cc:2258] T bd55b293540a4feb8daa06ea4220b5db P 3d0c878c4dd74021bd3406b93a11c7c4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:13:14.611649 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 28849
I20260807 18:13:14.612921 29731 ts_tablet_manager.cc:1905] T bd55b293540a4feb8daa06ea4220b5db P 3d0c878c4dd74021bd3406b93a11c7c4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:13:14.615386 29734 ts_tablet_manager.cc:1905] T bd55b293540a4feb8daa06ea4220b5db P 9b53136e1bd4469caee5c7f30bad8a38: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:13:14.618158 29731 ts_tablet_manager.cc:1918] T bd55b293540a4feb8daa06ea4220b5db P 3d0c878c4dd74021bd3406b93a11c7c4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:13:14.618312 29731 log.cc:1196] T bd55b293540a4feb8daa06ea4220b5db P 3d0c878c4dd74021bd3406b93a11c7c4: Deleting WAL directory at /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/ts-0/wal/wals/bd55b293540a4feb8daa06ea4220b5db
I20260807 18:13:14.619050 29731 ts_tablet_manager.cc:1939] T bd55b293540a4feb8daa06ea4220b5db P 3d0c878c4dd74021bd3406b93a11c7c4: Deleting consensus metadata
I20260807 18:13:14.620424 28578 catalog_manager.cc:4841] TS 3d0c878c4dd74021bd3406b93a11c7c4 (127.18.154.1:41283): tablet bd55b293540a4feb8daa06ea4220b5db (table test-table [id=82e685f489af40f3b3448215cbab10da]) successfully deleted
I20260807 18:13:14.621709 29738 tablet_replica.cc:330] T bd55b293540a4feb8daa06ea4220b5db P fde1306083f041638796f175e031fb30: stopping tablet replica
I20260807 18:13:14.622470 29738 raft_consensus.cc:2229] T bd55b293540a4feb8daa06ea4220b5db P fde1306083f041638796f175e031fb30 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:13:14.622884 29738 raft_consensus.cc:2258] T bd55b293540a4feb8daa06ea4220b5db P fde1306083f041638796f175e031fb30 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:13:14.624462 29734 ts_tablet_manager.cc:1918] T bd55b293540a4feb8daa06ea4220b5db P 9b53136e1bd4469caee5c7f30bad8a38: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:13:14.624624 29734 log.cc:1196] T bd55b293540a4feb8daa06ea4220b5db P 9b53136e1bd4469caee5c7f30bad8a38: Deleting WAL directory at /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/ts-2/wal/wals/bd55b293540a4feb8daa06ea4220b5db
I20260807 18:13:14.625054 29734 ts_tablet_manager.cc:1939] T bd55b293540a4feb8daa06ea4220b5db P 9b53136e1bd4469caee5c7f30bad8a38: Deleting consensus metadata
I20260807 18:13:14.625510 29738 ts_tablet_manager.cc:1905] T bd55b293540a4feb8daa06ea4220b5db P fde1306083f041638796f175e031fb30: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:13:14.626380 28579 catalog_manager.cc:4841] TS 9b53136e1bd4469caee5c7f30bad8a38 (127.18.154.3:46263): tablet bd55b293540a4feb8daa06ea4220b5db (table test-table [id=82e685f489af40f3b3448215cbab10da]) successfully deleted
I20260807 18:13:14.632588 29738 ts_tablet_manager.cc:1918] T bd55b293540a4feb8daa06ea4220b5db P fde1306083f041638796f175e031fb30: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:13:14.632753 29738 log.cc:1196] T bd55b293540a4feb8daa06ea4220b5db P fde1306083f041638796f175e031fb30: Deleting WAL directory at /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786126328808105-19048-0/minicluster-data/ts-1/wal/wals/bd55b293540a4feb8daa06ea4220b5db
I20260807 18:13:14.633229 29738 ts_tablet_manager.cc:1939] T bd55b293540a4feb8daa06ea4220b5db P fde1306083f041638796f175e031fb30: Deleting consensus metadata
I20260807 18:13:14.634531 28579 catalog_manager.cc:4841] TS fde1306083f041638796f175e031fb30 (127.18.154.2:42923): tablet bd55b293540a4feb8daa06ea4220b5db (table test-table [id=82e685f489af40f3b3448215cbab10da]) successfully deleted
I20260807 18:13:14.639158 29690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:14.612508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50012 (local address 127.18.154.1:41283)
0807 18:13:14.612665 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:14.612672 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:14.621908 (+ 9236us) server_negotiation.cc:407] Connection header received
0807 18:13:14.622102 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:14.622110 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:14.622199 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:14.622314 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:14.624666 (+ 2352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:14.625803 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:14.627740 (+ 1937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:14.628372 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:14.628469 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:14.632312 (+ 3843us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:13:14.632336 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:13:14.632363 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:13:14.632424 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:13:14.636088 (+ 3664us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:14.636902 (+ 814us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:14.636913 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:14.636917 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:14.637084 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:14.637516 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:14.637523 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:14.637528 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:14.637944 (+ 416us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:13:14.638279 (+ 335us) server_negotiation.cc:1035] Waiting for connection context
0807 18:13:14.638566 (+ 287us) server_negotiation.cc:299] Negotiation successful
0807 18:13:14.638785 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260807 18:13:14.639922 29124 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:13:14.770850 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 28849
I20260807 18:13:14.800273 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 29153
I20260807 18:13:14.814503 29685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:14.801131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37886 (local address 127.18.154.2:42923)
0807 18:13:14.801358 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:14.801364 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:14.801385 (+ 21us) server_negotiation.cc:407] Connection header received
0807 18:13:14.801458 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:14.801465 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:14.801573 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:14.801694 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:14.803271 (+ 1577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:14.804006 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:14.805291 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:14.805809 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:14.805951 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:14.808530 (+ 2579us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:13:14.808548 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:13:14.808565 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:13:14.808599 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:13:14.811575 (+ 2976us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:14.812302 (+ 727us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:14.812311 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:14.812313 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:14.812435 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:14.812819 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:14.812824 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:14.812828 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:14.813204 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:13:14.813509 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0807 18:13:14.813911 (+ 402us) server_negotiation.cc:299] Negotiation successful
0807 18:13:14.814162 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260807 18:13:14.815304 29324 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:13:14.912376 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 29153
I20260807 18:13:14.936734 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 29353
I20260807 18:13:14.950678 29744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:14.937568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35996 (local address 127.18.154.3:46263)
0807 18:13:14.938098 (+ 530us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:14.938102 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:14.938119 (+ 17us) server_negotiation.cc:407] Connection header received
0807 18:13:14.938178 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:14.938182 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:14.938253 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:14.938345 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:14.939680 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:14.940411 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:14.941919 (+ 1508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:14.942334 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:14.942429 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:14.944920 (+ 2491us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:13:14.944934 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:13:14.944952 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:13:14.944997 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:13:14.947942 (+ 2945us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:14.948566 (+ 624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:14.948573 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:14.948575 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:14.948697 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:14.949081 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:14.949084 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:14.949087 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:14.949481 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:13:14.949855 (+ 374us) server_negotiation.cc:1035] Waiting for connection context
0807 18:13:14.950125 (+ 270us) server_negotiation.cc:299] Negotiation successful
0807 18:13:14.950333 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":429,"thread_start_us":313,"threads_started":1}
I20260807 18:13:14.951372 29537 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:13:14.971050 29746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:14.970260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:34661 (local address 127.0.0.1:52830)
0807 18:13:14.970793 (+ 533us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:14.970887 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.62:34661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":393,"thread_start_us":284,"threads_started":1}
I20260807 18:13:14.972451 29746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:14.972059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:34661 (local address 127.0.0.1:52834)
0807 18:13:14.972233 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:14.972313 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.62:34661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260807 18:13:15.049139 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 29353
I20260807 18:13:15.072057 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 28411
I20260807 18:13:15.073432 28638 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:13:15.203611 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 28411
W20260807 18:13:15.233181 28855 connection.cc:535] server connection from 127.0.0.1:32934 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260807 18:13:15.233677 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 28544
I20260807 18:13:15.234833 28915 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:13:15.360248 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 28544
2026-08-07T18:13:15Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (10633 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126328808105-19048-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:13:15 dist-test-slave-gkw7 krb5kdc[29758](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:13:15 dist-test-slave-gkw7 krb5kdc[29758](info): set up 2 sockets
Aug 07 18:13:15 dist-test-slave-gkw7 krb5kdc[29758](info): commencing operation
krb5kdc: starting...
W20260807 18:13:16.650040 19048 mini_kdc.cc:121] Time spent starting KDC: real 1.234s user 0.000s sys 0.033s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126328808105-19048-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126328808105-19048-0/krb5kdc/test-user.keytab.
Aug 07 18:13:16 dist-test-slave-gkw7 krb5kdc[29758](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126396, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:13:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:13:16Z Disabled control of system clock
WARNING: no policy specified for oryx/127.18.154.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.154.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.154.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126328808105-19048-0/krb5kdc/oryx_127.18.154.62.keytab.
Entry for principal oryx/127.18.154.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126328808105-19048-0/krb5kdc/oryx_127.18.154.62.keytab.
I20260807 18:13:16.921834 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126328808105-19048-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126328808105-19048-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126328808105-19048-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.154.62:33361
--webserver_interface=127.18.154.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126328808105-19048-0/krb5kdc/oryx_127.18.154.62.keytab
--principal=oryx/127.18.154.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.154.20:34113
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.154.62:33361
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126328808105-19048-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126328808105-19048-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126328808105-19048-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:13:17.270797 29788 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:13:17.271122 29788 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:13:17.271299 29788 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:13:17.281684 29788 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:13:17.281836 29788 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260807 18:13:17.281914 29788 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:13:17.282025 29788 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:13:17.282119 29788 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:13:17.282205 29788 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:13:17.292716 29788 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.154.20:34113
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126328808105-19048-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126328808105-19048-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.154.62:33361
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126328808105-19048-0/krb5kdc/oryx_127.18.154.62.keytab
--principal=oryx/127.18.154.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.154.62:33361
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.154.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126328808105-19048-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:13:17.294220 29788 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:13:17.295643 29788 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:13:17.308236 29798 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:13:17.308305 29803 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:13:17.308348 29800 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:13:17.309360 29788 server_base.cc:1034] running on GCE node
I20260807 18:13:17.310232 29788 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:13:17.311740 29788 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:13:17.312999 29788 hybrid_clock.cc:648] HybridClock initialized: now 1786126397312919 us; error 57 us; skew 500 ppm
I20260807 18:13:17.313364 29788 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:13:17 dist-test-slave-gkw7 krb5kdc[29758](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126397, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.154.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:13:17.316764 29788 init.cc:377] Logged in from keytab as oryx/127.18.154.62@KRBTEST.COM (short username oryx)
I20260807 18:13:17.318317 29788 webserver.cc:460] Webserver started at http://127.18.154.62:45103/ using document root <none> and password file <none>
I20260807 18:13:17.319023 29788 fs_manager.cc:359] Metadata directory not provided
I20260807 18:13:17.319109 29788 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:13:17.319396 29788 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:13:17.322542 29788 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126328808105-19048-0/minicluster-data/master-0/data/instance:
uuid: "e61bfa7995fe4afcbcaf0939bac87929"
format_stamp: "Formatted at 2026-08-07 18:13:17 on dist-test-slave-gkw7"
server_key: "b1e0fdb0ad35cfb49e6d2525af688ab2"
server_key_iv: "be4cf290c6c067bc1e877dfa51a0f2ad"
server_key_version: "encryptionkey@0"
I20260807 18:13:17.323350 29788 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126328808105-19048-0/minicluster-data/master-0/wal/instance:
uuid: "e61bfa7995fe4afcbcaf0939bac87929"
format_stamp: "Formatted at 2026-08-07 18:13:17 on dist-test-slave-gkw7"
server_key: "b1e0fdb0ad35cfb49e6d2525af688ab2"
server_key_iv: "be4cf290c6c067bc1e877dfa51a0f2ad"
server_key_version: "encryptionkey@0"
I20260807 18:13:17.329052 29788 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 18:13:17.332794 29816 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:13:17.334262 29788 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:13:17.334401 29788 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126328808105-19048-0/minicluster-data/master-0/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126328808105-19048-0/minicluster-data/master-0/wal
uuid: "e61bfa7995fe4afcbcaf0939bac87929"
format_stamp: "Formatted at 2026-08-07 18:13:17 on dist-test-slave-gkw7"
server_key: "b1e0fdb0ad35cfb49e6d2525af688ab2"
server_key_iv: "be4cf290c6c067bc1e877dfa51a0f2ad"
server_key_version: "encryptionkey@0"
I20260807 18:13:17.334582 29788 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126328808105-19048-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126328808105-19048-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126328808105-19048-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:13:17.348483 29788 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:13:17.353134 29788 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:13:17.353374 29788 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:13:17.376471 29788 rpc_server.cc:307] RPC server started. Bound to: 127.18.154.62:33361
I20260807 18:13:17.376580 29911 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.154.62:33361 every 8 connection(s)
I20260807 18:13:17.378154 29788 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
I20260807 18:13:17.379284 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 29788
I20260807 18:13:17.379559 19048 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126328808105-19048-0/minicluster-data/master-0/wal/instance
I20260807 18:13:17.380223 19048 external_mini_cluster.cc:1428] Setting key 9bcad79a871fe59eb4470f0f8542a098
I20260807 18:13:17.383234 29912 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 07 18:13:17 dist-test-slave-gkw7 krb5kdc[29758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126396, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.18.154.62@KRBTEST.COM
I20260807 18:13:17.397486 29912 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e61bfa7995fe4afcbcaf0939bac87929: Bootstrap starting.
I20260807 18:13:17.400908 29912 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e61bfa7995fe4afcbcaf0939bac87929: Neither blocks nor log segments found. Creating new log.
I20260807 18:13:17.402128 29912 log.cc:824] T 00000000000000000000000000000000 P e61bfa7995fe4afcbcaf0939bac87929: Log is configured to *not* fsync() on all Append() calls
I20260807 18:13:17.405220 29912 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e61bfa7995fe4afcbcaf0939bac87929: No bootstrap required, opened a new log
I20260807 18:13:17.406147 29916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:17.382467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50440 (local address 127.18.154.62:33361)
0807 18:13:17.383392 (+ 925us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:17.383406 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:17.383455 (+ 49us) server_negotiation.cc:407] Connection header received
0807 18:13:17.385263 (+ 1808us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:17.385287 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:17.385795 (+ 508us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:17.386209 (+ 414us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:17.388210 (+ 2001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:17.389886 (+ 1676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:17.391723 (+ 1837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:17.392486 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:17.392626 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:17.397776 (+ 5150us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:13:17.397787 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:13:17.397805 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:13:17.397855 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:13:17.401408 (+ 3553us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:17.402256 (+ 848us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:17.402286 (+ 30us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:17.402299 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:17.402494 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:17.402953 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:17.402957 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:17.402960 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:17.403576 (+ 616us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:13:17.403908 (+ 332us) server_negotiation.cc:1035] Waiting for connection context
0807 18:13:17.404440 (+ 532us) server_negotiation.cc:299] Negotiation successful
0807 18:13:17.404898 (+ 458us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":633,"thread_start_us":426,"threads_started":1}
I20260807 18:13:17.413748 29912 raft_consensus.cc:348] T 00000000000000000000000000000000 P e61bfa7995fe4afcbcaf0939bac87929 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e61bfa7995fe4afcbcaf0939bac87929" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 33361 } }
I20260807 18:13:17.414166 29912 raft_consensus.cc:374] T 00000000000000000000000000000000 P e61bfa7995fe4afcbcaf0939bac87929 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:13:17.414278 29912 raft_consensus.cc:729] T 00000000000000000000000000000000 P e61bfa7995fe4afcbcaf0939bac87929 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e61bfa7995fe4afcbcaf0939bac87929, State: Initialized, Role: FOLLOWER
I20260807 18:13:17.414897 29912 consensus_queue.cc:260] T 00000000000000000000000000000000 P e61bfa7995fe4afcbcaf0939bac87929 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e61bfa7995fe4afcbcaf0939bac87929" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 33361 } }
I20260807 18:13:17.415155 29912 raft_consensus.cc:388] T 00000000000000000000000000000000 P e61bfa7995fe4afcbcaf0939bac87929 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:13:17.415311 29912 raft_consensus.cc:482] T 00000000000000000000000000000000 P e61bfa7995fe4afcbcaf0939bac87929 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:13:17.415500 29912 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e61bfa7995fe4afcbcaf0939bac87929 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:13:17.418145 29912 raft_consensus.cc:504] T 00000000000000000000000000000000 P e61bfa7995fe4afcbcaf0939bac87929 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e61bfa7995fe4afcbcaf0939bac87929" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 33361 } }
I20260807 18:13:17.418665 29912 leader_election.cc:302] T 00000000000000000000000000000000 P e61bfa7995fe4afcbcaf0939bac87929 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e61bfa7995fe4afcbcaf0939bac87929; no voters:
I20260807 18:13:17.419273 29912 leader_election.cc:288] T 00000000000000000000000000000000 P e61bfa7995fe4afcbcaf0939bac87929 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:13:17.419430 29920 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e61bfa7995fe4afcbcaf0939bac87929 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:13:17.420157 29920 raft_consensus.cc:686] T 00000000000000000000000000000000 P e61bfa7995fe4afcbcaf0939bac87929 [term 1 LEADER]: Becoming Leader. State: Replica: e61bfa7995fe4afcbcaf0939bac87929, State: Running, Role: LEADER
I20260807 18:13:17.420786 29920 consensus_queue.cc:237] T 00000000000000000000000000000000 P e61bfa7995fe4afcbcaf0939bac87929 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e61bfa7995fe4afcbcaf0939bac87929" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 33361 } }
I20260807 18:13:17.421479 29912 sys_catalog.cc:564] T 00000000000000000000000000000000 P e61bfa7995fe4afcbcaf0939bac87929 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:13:17.424852 29925 sys_catalog.cc:455] T 00000000000000000000000000000000 P e61bfa7995fe4afcbcaf0939bac87929 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e61bfa7995fe4afcbcaf0939bac87929. Latest consensus state: current_term: 1 leader_uuid: "e61bfa7995fe4afcbcaf0939bac87929" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e61bfa7995fe4afcbcaf0939bac87929" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 33361 } } }
I20260807 18:13:17.424826 29924 sys_catalog.cc:455] T 00000000000000000000000000000000 P e61bfa7995fe4afcbcaf0939bac87929 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e61bfa7995fe4afcbcaf0939bac87929" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e61bfa7995fe4afcbcaf0939bac87929" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 33361 } } }
I20260807 18:13:17.425590 29924 sys_catalog.cc:458] T 00000000000000000000000000000000 P e61bfa7995fe4afcbcaf0939bac87929 [sys.catalog]: This master's current role is: LEADER
I20260807 18:13:17.427189 29936 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:13:17.427157 29925 sys_catalog.cc:458] T 00000000000000000000000000000000 P e61bfa7995fe4afcbcaf0939bac87929 [sys.catalog]: This master's current role is: LEADER
I20260807 18:13:17.432313 29936 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:13:17.441198 29936 catalog_manager.cc:1347] Generated new cluster ID: f5eb815203be42928114706b9ee5151c
I20260807 18:13:17.441337 29936 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:13:17.453277 29936 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:13:17.455578 29936 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:13:17.467063 29936 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e61bfa7995fe4afcbcaf0939bac87929: Generated new TSK 0
I20260807 18:13:17.468132 29936 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.18.154.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.154.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.154.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126328808105-19048-0/krb5kdc/oryx_127.18.154.1.keytab.
Entry for principal oryx/127.18.154.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126328808105-19048-0/krb5kdc/oryx_127.18.154.1.keytab.
I20260807 18:13:17.549242 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126328808105-19048-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126328808105-19048-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126328808105-19048-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126328808105-19048-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.154.1:0
--local_ip_for_outbound_sockets=127.18.154.1
--webserver_interface=127.18.154.1
--webserver_port=0
--tserver_master_addrs=127.18.154.62:33361
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126328808105-19048-0/krb5kdc/oryx_127.18.154.1.keytab
--principal=oryx/127.18.154.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.154.20:34113
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126328808105-19048-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126328808105-19048-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126328808105-19048-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:13:17.906430 29953 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:13:17.906812 29953 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:13:17.906996 29953 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:13:17.914603 29953 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:13:17.914767 29953 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:13:17.914911 29953 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 18:13:17.915060 29953 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.154.1
I20260807 18:13:17.925679 29953 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.154.20:34113
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126328808105-19048-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126328808105-19048-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126328808105-19048-0/krb5kdc/oryx_127.18.154.1.keytab
--principal=oryx/127.18.154.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.154.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126328808105-19048-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.154.1
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.18.154.62:33361
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.154.1
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126328808105-19048-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:13:17.927239 29953 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:13:17.928574 29953 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:13:17.942814 29963 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:13:17.942902 29965 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:13:17.942902 29967 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:13:17.943868 29953 server_base.cc:1034] running on GCE node
I20260807 18:13:17.944525 29953 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:13:17.945602 29953 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:13:17.946879 29953 hybrid_clock.cc:648] HybridClock initialized: now 1786126397946794 us; error 61 us; skew 500 ppm
I20260807 18:13:17.947248 29953 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:13:17 dist-test-slave-gkw7 krb5kdc[29758](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126397, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.154.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:13:17.955204 29953 init.cc:377] Logged in from keytab as oryx/127.18.154.1@KRBTEST.COM (short username oryx)
I20260807 18:13:17.956903 29953 webserver.cc:460] Webserver started at http://127.18.154.1:36839/ using document root <none> and password file <none>
I20260807 18:13:17.957715 29953 fs_manager.cc:359] Metadata directory not provided
I20260807 18:13:17.957814 29953 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:13:17.958108 29953 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:13:17.961062 29953 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126328808105-19048-0/minicluster-data/ts-0/data/instance:
uuid: "f364d0e1a3f14b55829f9ee56ac06792"
format_stamp: "Formatted at 2026-08-07 18:13:17 on dist-test-slave-gkw7"
server_key: "126b9f189e443e9832463a1292f4d762"
server_key_iv: "d4118fc5c04ae3e0062c763681f3903b"
server_key_version: "encryptionkey@0"
I20260807 18:13:17.961956 29953 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126328808105-19048-0/minicluster-data/ts-0/wal/instance:
uuid: "f364d0e1a3f14b55829f9ee56ac06792"
format_stamp: "Formatted at 2026-08-07 18:13:17 on dist-test-slave-gkw7"
server_key: "126b9f189e443e9832463a1292f4d762"
server_key_iv: "d4118fc5c04ae3e0062c763681f3903b"
server_key_version: "encryptionkey@0"
I20260807 18:13:17.967489 29953 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260807 18:13:17.971138 29977 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:13:17.972522 29953 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 18:13:17.972663 29953 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126328808105-19048-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126328808105-19048-0/minicluster-data/ts-0/wal
uuid: "f364d0e1a3f14b55829f9ee56ac06792"
format_stamp: "Formatted at 2026-08-07 18:13:17 on dist-test-slave-gkw7"
server_key: "126b9f189e443e9832463a1292f4d762"
server_key_iv: "d4118fc5c04ae3e0062c763681f3903b"
server_key_version: "encryptionkey@0"
I20260807 18:13:17.972857 29953 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126328808105-19048-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126328808105-19048-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126328808105-19048-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:13:17.993403 29953 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:13:17.998148 29953 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:13:17.998386 29953 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:13:18.001183 29953 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:13:18.001273 29953 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:13:18.001573 29953 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:13:18.001698 29953 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 18:13:18 dist-test-slave-gkw7 krb5kdc[29758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126397, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.154.1@KRBTEST.COM for oryx/127.18.154.62@KRBTEST.COM
I20260807 18:13:18.025818 29989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:18.001683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.1:41723 (local address 127.18.154.62:33361)
0807 18:13:18.002217 (+ 534us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:18.002224 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:18.003537 (+ 1313us) server_negotiation.cc:407] Connection header received
0807 18:13:18.005881 (+ 2344us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:18.005890 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:18.006024 (+ 134us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:18.006157 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:18.008591 (+ 2434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:18.009729 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:18.012485 (+ 2756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:18.012913 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:18.013066 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:18.018384 (+ 5318us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:13:18.018389 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:13:18.018392 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:13:18.018431 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:13:18.021271 (+ 2840us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:18.022598 (+ 1327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:18.022607 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:18.022613 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:18.022850 (+ 237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:18.023392 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:18.023400 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:18.023406 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:18.023922 (+ 516us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:13:18.024121 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0807 18:13:18.024884 (+ 763us) server_negotiation.cc:299] Negotiation successful
0807 18:13:18.025225 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":403,"thread_start_us":274,"threads_started":1}
I20260807 18:13:18.026681 29991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:18.002410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:33361 (local address 127.18.154.1:41723)
0807 18:13:18.003318 (+ 908us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:18.003369 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0807 18:13:18.005523 (+ 2154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:13:18.006441 (+ 918us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:13:18.006450 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:13:18.007274 (+ 824us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:13:18.008352 (+ 1078us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:18.008392 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:18.009898 (+ 1506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:18.009905 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:18.012301 (+ 2396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:18.012315 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:18.013223 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:18.013233 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:18.013490 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:18.014375 (+ 885us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:13:18.014425 (+ 50us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:13:18.018108 (+ 3683us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:13:18.021440 (+ 3332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:13:18.021450 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:13:18.021471 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:13:18.022413 (+ 942us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:13:18.022997 (+ 584us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:13:18.023002 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:13:18.023005 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:13:18.023254 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:13:18.024062 (+ 808us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:13:18.024069 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:13:18.024468 (+ 399us) client_negotiation.cc:769] Sending connection context
0807 18:13:18.024707 (+ 239us) client_negotiation.cc:240] Negotiation successful
0807 18:13:18.025127 (+ 420us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":626,"thread_start_us":381,"threads_started":1}
I20260807 18:13:18.052079 29989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:18.034715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.1:42957 (local address 127.18.154.62:33361)
0807 18:13:18.034986 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:18.034995 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:18.035025 (+ 30us) server_negotiation.cc:407] Connection header received
0807 18:13:18.035253 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:18.035263 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:18.035371 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:18.035526 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:18.036575 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:18.038087 (+ 1512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:18.040921 (+ 2834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:18.041599 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:18.041763 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:18.044866 (+ 3103us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:13:18.044872 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:13:18.044875 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:13:18.044911 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:13:18.048617 (+ 3706us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:18.049885 (+ 1268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:18.049891 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:18.049895 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:18.050045 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:18.050521 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:18.050525 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:18.050527 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:18.050841 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:13:18.050968 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0807 18:13:18.051601 (+ 633us) server_negotiation.cc:299] Negotiation successful
0807 18:13:18.051841 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260807 18:13:18.052668 30081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:18.034267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:33361 (local address 127.18.154.1:42957)
0807 18:13:18.034914 (+ 647us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:18.034935 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 18:13:18.035092 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:13:18.035572 (+ 480us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:13:18.035576 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:13:18.036084 (+ 508us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:13:18.036403 (+ 319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:18.036415 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:18.038309 (+ 1894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:18.038315 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:18.040704 (+ 2389us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:18.040731 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:18.042070 (+ 1339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:18.042075 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:18.042277 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:18.043247 (+ 970us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:13:18.043273 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:13:18.044545 (+ 1272us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:13:18.048812 (+ 4267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:13:18.048820 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:13:18.048827 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:13:18.049730 (+ 903us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:13:18.050182 (+ 452us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:13:18.050189 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:13:18.050196 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:13:18.050396 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:13:18.050998 (+ 602us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:13:18.051007 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:13:18.051288 (+ 281us) client_negotiation.cc:769] Sending connection context
0807 18:13:18.051495 (+ 207us) client_negotiation.cc:240] Negotiation successful
0807 18:13:18.052001 (+ 506us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":474,"thread_start_us":514,"threads_started":1}
I20260807 18:13:18.061614 29953 rpc_server.cc:307] RPC server started. Bound to: 127.18.154.1:46809
I20260807 18:13:18.061646 30158 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.154.1:46809 every 8 connection(s)
I20260807 18:13:18.062928 29953 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126328808105-19048-0/minicluster-data/ts-0/data/info.pb
I20260807 18:13:18.065279 30160 heartbeater.cc:347] Connected to a master server at 127.18.154.62:33361
I20260807 18:13:18.065609 30160 heartbeater.cc:464] Registering TS with master...
I20260807 18:13:18.066313 30160 heartbeater.cc:511] Master 127.18.154.62:33361 requested a full tablet report, sending...
I20260807 18:13:18.068673 29848 ts_manager.cc:194] Registered new tserver with Master: f364d0e1a3f14b55829f9ee56ac06792 (127.18.154.1:46809)
I20260807 18:13:18.069214 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 29953
I20260807 18:13:18.069459 19048 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126328808105-19048-0/minicluster-data/ts-0/wal/instance
I20260807 18:13:18.070204 19048 external_mini_cluster.cc:1428] Setting key 3841b532b46e14b2186c1038b8defd48
I20260807 18:13:18.070430 29848 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.18.154.1@KRBTEST.COM'} at 127.18.154.1:41723
WARNING: no policy specified for oryx/127.18.154.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.154.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.154.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126328808105-19048-0/krb5kdc/oryx_127.18.154.2.keytab.
Entry for principal oryx/127.18.154.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126328808105-19048-0/krb5kdc/oryx_127.18.154.2.keytab.
I20260807 18:13:18.131002 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126328808105-19048-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126328808105-19048-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126328808105-19048-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126328808105-19048-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.154.2:0
--local_ip_for_outbound_sockets=127.18.154.2
--webserver_interface=127.18.154.2
--webserver_port=0
--tserver_master_addrs=127.18.154.62:33361
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126328808105-19048-0/krb5kdc/oryx_127.18.154.2.keytab
--principal=oryx/127.18.154.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.154.20:34113
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126328808105-19048-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126328808105-19048-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126328808105-19048-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:13:18.473412 30171 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:13:18.473848 30171 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:13:18.474026 30171 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:13:18.481606 30171 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:13:18.481722 30171 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:13:18.481897 30171 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 18:13:18.482023 30171 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.154.2
I20260807 18:13:18.492097 30171 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.154.20:34113
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126328808105-19048-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126328808105-19048-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126328808105-19048-0/krb5kdc/oryx_127.18.154.2.keytab
--principal=oryx/127.18.154.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.154.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126328808105-19048-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.154.2
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.18.154.62:33361
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.154.2
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126328808105-19048-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:13:18.493471 30171 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:13:18.494938 30171 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:13:18.509073 30181 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:13:18.509047 30184 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:13:18.508993 30182 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:13:18.510059 30171 server_base.cc:1034] running on GCE node
I20260807 18:13:18.510695 30171 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:13:18.511655 30171 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:13:18.512892 30171 hybrid_clock.cc:648] HybridClock initialized: now 1786126398512839 us; error 32 us; skew 500 ppm
I20260807 18:13:18.513211 30171 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:13:18 dist-test-slave-gkw7 krb5kdc[29758](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126398, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.154.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:13:18.516670 30171 init.cc:377] Logged in from keytab as oryx/127.18.154.2@KRBTEST.COM (short username oryx)
I20260807 18:13:18.518333 30171 webserver.cc:460] Webserver started at http://127.18.154.2:38315/ using document root <none> and password file <none>
I20260807 18:13:18.519025 30171 fs_manager.cc:359] Metadata directory not provided
I20260807 18:13:18.519114 30171 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:13:18.519394 30171 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:13:18.522836 30171 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126328808105-19048-0/minicluster-data/ts-1/data/instance:
uuid: "dd13563f5c834551b7f1830266335364"
format_stamp: "Formatted at 2026-08-07 18:13:18 on dist-test-slave-gkw7"
server_key: "cc71861344eb127d1795a6276a7c1e2a"
server_key_iv: "5c83ddcd9ea56bb8eb442dbeadb9a39a"
server_key_version: "encryptionkey@0"
I20260807 18:13:18.523607 30171 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126328808105-19048-0/minicluster-data/ts-1/wal/instance:
uuid: "dd13563f5c834551b7f1830266335364"
format_stamp: "Formatted at 2026-08-07 18:13:18 on dist-test-slave-gkw7"
server_key: "cc71861344eb127d1795a6276a7c1e2a"
server_key_iv: "5c83ddcd9ea56bb8eb442dbeadb9a39a"
server_key_version: "encryptionkey@0"
I20260807 18:13:18.528890 30171 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.005s
I20260807 18:13:18.532482 30192 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:13:18.534106 30171 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:13:18.534293 30171 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126328808105-19048-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126328808105-19048-0/minicluster-data/ts-1/wal
uuid: "dd13563f5c834551b7f1830266335364"
format_stamp: "Formatted at 2026-08-07 18:13:18 on dist-test-slave-gkw7"
server_key: "cc71861344eb127d1795a6276a7c1e2a"
server_key_iv: "5c83ddcd9ea56bb8eb442dbeadb9a39a"
server_key_version: "encryptionkey@0"
I20260807 18:13:18.534554 30171 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126328808105-19048-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126328808105-19048-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126328808105-19048-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:13:18.556747 30171 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:13:18.561403 30171 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:13:18.561725 30171 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:13:18.564358 30171 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:13:18.564477 30171 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:13:18.564710 30171 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:13:18.564810 30171 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 18:13:18 dist-test-slave-gkw7 krb5kdc[29758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126398, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.154.2@KRBTEST.COM for oryx/127.18.154.62@KRBTEST.COM
I20260807 18:13:18.590874 30213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:18.565189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.2:49189 (local address 127.18.154.62:33361)
0807 18:13:18.565722 (+ 533us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:18.565729 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:18.567011 (+ 1282us) server_negotiation.cc:407] Connection header received
0807 18:13:18.570284 (+ 3273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:18.570292 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:18.570387 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:18.570535 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:18.572964 (+ 2429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:18.574432 (+ 1468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:18.576645 (+ 2213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:18.577461 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:18.577650 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:18.582899 (+ 5249us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:13:18.582909 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:13:18.582917 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:13:18.582973 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:13:18.586419 (+ 3446us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:18.587662 (+ 1243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:18.587671 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:18.587677 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:18.587864 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:18.588587 (+ 723us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:18.588592 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:18.588595 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:18.588916 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:13:18.589119 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0807 18:13:18.590189 (+ 1070us) server_negotiation.cc:299] Negotiation successful
0807 18:13:18.590446 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":401,"thread_start_us":271,"threads_started":1}
I20260807 18:13:18.592576 30214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:18.565883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:33361 (local address 127.18.154.2:49189)
0807 18:13:18.566815 (+ 932us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:18.566866 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0807 18:13:18.569893 (+ 3027us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:13:18.570861 (+ 968us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:13:18.570876 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:13:18.571738 (+ 862us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:13:18.572731 (+ 993us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:18.572766 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:18.574614 (+ 1848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:18.574622 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:18.576473 (+ 1851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:18.576495 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:18.577775 (+ 1280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:18.577785 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:18.578096 (+ 311us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:18.579010 (+ 914us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:13:18.579048 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:13:18.582628 (+ 3580us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:13:18.586616 (+ 3988us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:13:18.586631 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:13:18.586650 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:13:18.587423 (+ 773us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:13:18.588028 (+ 605us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:13:18.588031 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:13:18.588034 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:13:18.588388 (+ 354us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:13:18.589095 (+ 707us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:13:18.589103 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:13:18.589636 (+ 533us) client_negotiation.cc:769] Sending connection context
0807 18:13:18.590090 (+ 454us) client_negotiation.cc:240] Negotiation successful
0807 18:13:18.590732 (+ 642us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":616,"thread_start_us":362,"threads_started":1}
I20260807 18:13:18.616030 30319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:18.599022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:33361 (local address 127.18.154.2:56553)
0807 18:13:18.599800 (+ 778us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:18.599817 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 18:13:18.599942 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:13:18.600382 (+ 440us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:13:18.600386 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:13:18.600842 (+ 456us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:13:18.601135 (+ 293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:18.601146 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:18.602807 (+ 1661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:18.602815 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:18.604711 (+ 1896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:18.604740 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:18.605786 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:18.605793 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:18.605975 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:18.606706 (+ 731us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:13:18.606725 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:13:18.608093 (+ 1368us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:13:18.612705 (+ 4612us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:13:18.612717 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:13:18.612722 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:13:18.613231 (+ 509us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:13:18.613714 (+ 483us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:13:18.613720 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:13:18.613724 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:13:18.613881 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:13:18.614583 (+ 702us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:13:18.614588 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:13:18.614855 (+ 267us) client_negotiation.cc:769] Sending connection context
0807 18:13:18.615103 (+ 248us) client_negotiation.cc:240] Negotiation successful
0807 18:13:18.615504 (+ 401us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":593,"thread_start_us":401,"threads_started":1}
I20260807 18:13:18.616210 30171 rpc_server.cc:307] RPC server started. Bound to: 127.18.154.2:36977
I20260807 18:13:18.616274 30375 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.154.2:36977 every 8 connection(s)
I20260807 18:13:18.617038 30213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:18.598952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.2:56553 (local address 127.18.154.62:33361)
0807 18:13:18.599182 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:18.599189 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:18.599910 (+ 721us) server_negotiation.cc:407] Connection header received
0807 18:13:18.600067 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:18.600076 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:18.600196 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:18.600334 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:18.601291 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:18.602634 (+ 1343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:18.604967 (+ 2333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:18.605504 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:18.605718 (+ 214us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:18.608327 (+ 2609us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:13:18.608332 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:13:18.608336 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:13:18.608369 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:13:18.612489 (+ 4120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:18.613389 (+ 900us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:18.613393 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:18.613396 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:18.613517 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:18.614008 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:18.614017 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:18.614021 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:18.614436 (+ 415us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:13:18.614611 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0807 18:13:18.616625 (+ 2014us) server_negotiation.cc:299] Negotiation successful
0807 18:13:18.616810 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260807 18:13:18.617733 30171 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126328808105-19048-0/minicluster-data/ts-1/data/info.pb
I20260807 18:13:18.619762 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 30171
I20260807 18:13:18.619969 19048 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126328808105-19048-0/minicluster-data/ts-1/wal/instance
I20260807 18:13:18.620560 19048 external_mini_cluster.cc:1428] Setting key e65bac396ec138573dbf8c0d40563400
I20260807 18:13:18.620348 30376 heartbeater.cc:347] Connected to a master server at 127.18.154.62:33361
I20260807 18:13:18.621717 30376 heartbeater.cc:464] Registering TS with master...
I20260807 18:13:18.622524 30376 heartbeater.cc:511] Master 127.18.154.62:33361 requested a full tablet report, sending...
I20260807 18:13:18.624274 29848 ts_manager.cc:194] Registered new tserver with Master: dd13563f5c834551b7f1830266335364 (127.18.154.2:36977)
I20260807 18:13:18.625626 29848 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.18.154.2@KRBTEST.COM'} at 127.18.154.2:49189
WARNING: no policy specified for oryx/127.18.154.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.154.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.154.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126328808105-19048-0/krb5kdc/oryx_127.18.154.3.keytab.
Entry for principal oryx/127.18.154.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126328808105-19048-0/krb5kdc/oryx_127.18.154.3.keytab.
I20260807 18:13:18.680755 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126328808105-19048-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126328808105-19048-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126328808105-19048-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126328808105-19048-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.154.3:0
--local_ip_for_outbound_sockets=127.18.154.3
--webserver_interface=127.18.154.3
--webserver_port=0
--tserver_master_addrs=127.18.154.62:33361
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126328808105-19048-0/krb5kdc/oryx_127.18.154.3.keytab
--principal=oryx/127.18.154.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.154.20:34113
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126328808105-19048-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126328808105-19048-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126328808105-19048-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:13:19.028060 30383 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:13:19.028417 30383 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:13:19.028586 30383 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:13:19.036478 30383 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:13:19.036646 30383 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:13:19.036813 30383 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 18:13:19.036955 30383 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.154.3
I20260807 18:13:19.047297 30383 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.154.20:34113
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126328808105-19048-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126328808105-19048-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126328808105-19048-0/krb5kdc/oryx_127.18.154.3.keytab
--principal=oryx/127.18.154.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.154.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126328808105-19048-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.154.3
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.18.154.62:33361
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.154.3
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126328808105-19048-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:13:19.048703 30383 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:13:19.050158 30383 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:13:19.064512 30389 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:13:19.064936 30391 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:13:19.064630 30393 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:13:19.065876 30383 server_base.cc:1034] running on GCE node
I20260807 18:13:19.066540 30383 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:13:19.067569 30383 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:13:19.068842 30383 hybrid_clock.cc:648] HybridClock initialized: now 1786126399068758 us; error 55 us; skew 500 ppm
I20260807 18:13:19.069223 30383 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:13:19 dist-test-slave-gkw7 krb5kdc[29758](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126399, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.154.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:13:19.072726 30383 init.cc:377] Logged in from keytab as oryx/127.18.154.3@KRBTEST.COM (short username oryx)
I20260807 18:13:19.074160 30160 heartbeater.cc:503] Master 127.18.154.62:33361 was elected leader, sending a full tablet report...
I20260807 18:13:19.074486 30383 webserver.cc:460] Webserver started at http://127.18.154.3:40907/ using document root <none> and password file <none>
I20260807 18:13:19.075402 30383 fs_manager.cc:359] Metadata directory not provided
I20260807 18:13:19.075539 30383 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:13:19.075913 30383 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:13:19.079226 30383 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126328808105-19048-0/minicluster-data/ts-2/data/instance:
uuid: "a7039556d1bb4f9991853edeb64b2a62"
format_stamp: "Formatted at 2026-08-07 18:13:19 on dist-test-slave-gkw7"
server_key: "2ffa474b77e18e591012db192e9e7542"
server_key_iv: "210767a1fa21d5f111c57d7f20135aa1"
server_key_version: "encryptionkey@0"
I20260807 18:13:19.080091 30383 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126328808105-19048-0/minicluster-data/ts-2/wal/instance:
uuid: "a7039556d1bb4f9991853edeb64b2a62"
format_stamp: "Formatted at 2026-08-07 18:13:19 on dist-test-slave-gkw7"
server_key: "2ffa474b77e18e591012db192e9e7542"
server_key_iv: "210767a1fa21d5f111c57d7f20135aa1"
server_key_version: "encryptionkey@0"
I20260807 18:13:19.088819 30383 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.009s sys 0.000s
I20260807 18:13:19.093322 30403 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:13:19.095036 30383 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.003s
I20260807 18:13:19.095245 30383 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126328808105-19048-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126328808105-19048-0/minicluster-data/ts-2/wal
uuid: "a7039556d1bb4f9991853edeb64b2a62"
format_stamp: "Formatted at 2026-08-07 18:13:19 on dist-test-slave-gkw7"
server_key: "2ffa474b77e18e591012db192e9e7542"
server_key_iv: "210767a1fa21d5f111c57d7f20135aa1"
server_key_version: "encryptionkey@0"
I20260807 18:13:19.095541 30383 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126328808105-19048-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126328808105-19048-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126328808105-19048-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:13:19.123845 30383 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:13:19.128664 30383 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:13:19.128957 30383 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:13:19.131980 30383 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:13:19.132102 30383 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:13:19.132287 30383 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:13:19.132385 30383 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 18:13:19 dist-test-slave-gkw7 krb5kdc[29758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126399, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.154.3@KRBTEST.COM for oryx/127.18.154.62@KRBTEST.COM
I20260807 18:13:19.162120 30420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:19.132974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.3:58079 (local address 127.18.154.62:33361)
0807 18:13:19.133516 (+ 542us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:19.133523 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:19.135112 (+ 1589us) server_negotiation.cc:407] Connection header received
0807 18:13:19.138440 (+ 3328us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:19.138448 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:19.138561 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:19.138758 (+ 197us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:19.142248 (+ 3490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:19.143533 (+ 1285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:19.146509 (+ 2976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:19.147004 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:19.147183 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:19.152132 (+ 4949us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:13:19.152139 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:13:19.152148 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:13:19.152210 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:13:19.157300 (+ 5090us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:19.158521 (+ 1221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:19.158531 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:19.158537 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:19.158753 (+ 216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:19.159431 (+ 678us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:19.159437 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:19.159440 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:19.159896 (+ 456us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:13:19.160071 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0807 18:13:19.161251 (+ 1180us) server_negotiation.cc:299] Negotiation successful
0807 18:13:19.161625 (+ 374us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":405,"thread_start_us":248,"threads_started":1}
I20260807 18:13:19.163256 30421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:19.133804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:33361 (local address 127.18.154.3:58079)
0807 18:13:19.134858 (+ 1054us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:19.134924 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0807 18:13:19.138022 (+ 3098us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:13:19.139505 (+ 1483us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:13:19.139522 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:13:19.140578 (+ 1056us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:13:19.141967 (+ 1389us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:19.142009 (+ 42us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:19.143755 (+ 1746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:19.143768 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:19.146304 (+ 2536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:19.146323 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:19.147317 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:19.147327 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:19.147593 (+ 266us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:19.148413 (+ 820us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:13:19.148439 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:13:19.151730 (+ 3291us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:13:19.157739 (+ 6009us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:13:19.157749 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:13:19.157765 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:13:19.158375 (+ 610us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:13:19.158871 (+ 496us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:13:19.158877 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:13:19.158881 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:13:19.159297 (+ 416us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:13:19.160039 (+ 742us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:13:19.160048 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:13:19.160690 (+ 642us) client_negotiation.cc:769] Sending connection context
0807 18:13:19.161095 (+ 405us) client_negotiation.cc:240] Negotiation successful
0807 18:13:19.161728 (+ 633us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":731,"thread_start_us":440,"threads_started":1}
I20260807 18:13:19.190485 30420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:19.172807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.3:41089 (local address 127.18.154.62:33361)
0807 18:13:19.173049 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:19.173058 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:19.173685 (+ 627us) server_negotiation.cc:407] Connection header received
0807 18:13:19.173918 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:19.173925 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:19.174039 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:19.174153 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:19.175824 (+ 1671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:19.176853 (+ 1029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:19.179509 (+ 2656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:19.180040 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:19.180216 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:19.182730 (+ 2514us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:13:19.182740 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:13:19.182748 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:13:19.182816 (+ 68us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:13:19.187018 (+ 4202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:19.188188 (+ 1170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:19.188193 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:19.188196 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:19.188336 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:19.188777 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:19.188782 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:19.188791 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:19.189090 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:13:19.189283 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0807 18:13:19.189803 (+ 520us) server_negotiation.cc:299] Negotiation successful
0807 18:13:19.190074 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260807 18:13:19.190600 30516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:19.172801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:33361 (local address 127.18.154.3:41089)
0807 18:13:19.173563 (+ 762us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:19.173584 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 18:13:19.173771 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:13:19.174225 (+ 454us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:13:19.174233 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:13:19.175088 (+ 855us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:13:19.175586 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:19.175604 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:19.177016 (+ 1412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:19.177025 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:19.179318 (+ 2293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:19.179345 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:19.180351 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:19.180365 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:19.180660 (+ 295us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:19.181522 (+ 862us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:13:19.181614 (+ 92us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:13:19.182483 (+ 869us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:13:19.187199 (+ 4716us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:13:19.187208 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:13:19.187218 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:13:19.187992 (+ 774us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:13:19.188481 (+ 489us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:13:19.188486 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:13:19.188491 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:13:19.188645 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:13:19.189222 (+ 577us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:13:19.189227 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:13:19.189453 (+ 226us) client_negotiation.cc:769] Sending connection context
0807 18:13:19.189688 (+ 235us) client_negotiation.cc:240] Negotiation successful
0807 18:13:19.190083 (+ 395us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":570,"thread_start_us":411,"threads_started":1}
I20260807 18:13:19.201231 30383 rpc_server.cc:307] RPC server started. Bound to: 127.18.154.3:37679
I20260807 18:13:19.201267 30586 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.154.3:37679 every 8 connection(s)
I20260807 18:13:19.202703 30383 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126328808105-19048-0/minicluster-data/ts-2/data/info.pb
I20260807 18:13:19.205015 30588 heartbeater.cc:347] Connected to a master server at 127.18.154.62:33361
I20260807 18:13:19.205412 30588 heartbeater.cc:464] Registering TS with master...
I20260807 18:13:19.206221 30588 heartbeater.cc:511] Master 127.18.154.62:33361 requested a full tablet report, sending...
I20260807 18:13:19.207661 29848 ts_manager.cc:194] Registered new tserver with Master: a7039556d1bb4f9991853edeb64b2a62 (127.18.154.3:37679)
I20260807 18:13:19.208503 29848 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.18.154.3@KRBTEST.COM'} at 127.18.154.3:58079
I20260807 18:13:19.209137 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 30383
I20260807 18:13:19.209405 19048 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126328808105-19048-0/minicluster-data/ts-2/wal/instance
I20260807 18:13:19.209899 19048 external_mini_cluster.cc:1428] Setting key 05d06d615dcba4733a38f13304b45f68
I20260807 18:13:19.213008 19048 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 18:13:19 dist-test-slave-gkw7 krb5kdc[29758](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126399, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 18:13:19 dist-test-slave-gkw7 krb5kdc[29758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.18.154.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:13:19 dist-test-slave-gkw7 krb5kdc[29758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.18.154.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:13:19.628604 30376 heartbeater.cc:503] Master 127.18.154.62:33361 was elected leader, sending a full tablet report...
I20260807 18:13:20.211683 30588 heartbeater.cc:503] Master 127.18.154.62:33361 was elected leader, sending a full tablet report...
W20260807 18:13:20.483107 30598 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.215s user 0.007s sys 0.014s
W20260807 18:13:20.484498 30420 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:13:19.261996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50442 (local address 127.18.154.62:33361)
0807 18:13:19.262250 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:19.262259 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:19.262521 (+ 262us) server_negotiation.cc:407] Connection header received
0807 18:13:19.262780 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:19.262786 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:19.262864 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:19.262975 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:19.264349 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:19.265093 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:19.266586 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:19.267030 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:19.267184 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:20.484153 (+1216969us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50442: BlockingRecv error: recv got EOF from 127.0.0.1:50442 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
Aug 07 18:13:20 dist-test-slave-gkw7 krb5kdc[29758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126399, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.18.154.62@KRBTEST.COM
I20260807 18:13:20.504225 30420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:20.489704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50446 (local address 127.18.154.62:33361)
0807 18:13:20.489936 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:20.489943 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:20.490183 (+ 240us) server_negotiation.cc:407] Connection header received
0807 18:13:20.490397 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:20.490402 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:20.490481 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:20.490594 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:20.491876 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:20.492614 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:20.494004 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:20.494502 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:20.494647 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:20.498601 (+ 3954us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:13:20.498610 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:13:20.498619 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:13:20.498653 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:13:20.501387 (+ 2734us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:20.502168 (+ 781us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:20.502174 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:20.502177 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:20.502310 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:20.502698 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:20.502703 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:20.502707 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:20.502979 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:13:20.503302 (+ 323us) server_negotiation.cc:1035] Waiting for connection context
0807 18:13:20.503604 (+ 302us) server_negotiation.cc:299] Negotiation successful
0807 18:13:20.503875 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260807 18:13:20.508751 29848 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50446:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 18:13:20.516547 29848 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:13:20.553663 30629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:20.532897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60300 (local address 127.18.154.1:46809)
0807 18:13:20.545138 (+ 12241us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:20.545145 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:20.545716 (+ 571us) server_negotiation.cc:407] Connection header received
0807 18:13:20.546013 (+ 297us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:20.546021 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:20.546248 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:20.546407 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:13:20.547204 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:20.548373 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:20.551033 (+ 2660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:20.552502 (+ 1469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:20.552629 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:20.552767 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0807 18:13:20.553201 (+ 434us) server_negotiation.cc:299] Negotiation successful
0807 18:13:20.553323 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12092,"thread_start_us":11943,"threads_started":1}
I20260807 18:13:20.553802 30627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:20.532633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.1:46809 (local address 127.0.0.1:60300)
0807 18:13:20.545592 (+ 12959us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:20.545621 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0807 18:13:20.545798 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:13:20.546652 (+ 854us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:13:20.546657 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:13:20.546681 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:13:20.547011 (+ 330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:20.547020 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:20.548522 (+ 1502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:20.548528 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:20.550852 (+ 2324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:20.550865 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:20.552705 (+ 1840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:20.552712 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:20.552906 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:20.552958 (+ 52us) client_negotiation.cc:769] Sending connection context
0807 18:13:20.553091 (+ 133us) client_negotiation.cc:240] Negotiation successful
0807 18:13:20.553351 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12801,"mutex_wait_us":38,"thread_start_us":12651,"threads_started":1}
I20260807 18:13:20.554468 30621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:20.529966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.3:37679 (local address 127.0.0.1:45198)
0807 18:13:20.543346 (+ 13380us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:20.543419 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0807 18:13:20.543667 (+ 248us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:13:20.544831 (+ 1164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:13:20.544843 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:13:20.544884 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:13:20.545441 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:20.545455 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:20.547534 (+ 2079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:20.547540 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:20.550385 (+ 2845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:20.550400 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:20.553696 (+ 3296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:20.553702 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:20.553892 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:20.553940 (+ 48us) client_negotiation.cc:769] Sending connection context
0807 18:13:20.554027 (+ 87us) client_negotiation.cc:240] Negotiation successful
0807 18:13:20.554166 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13026,"mutex_wait_us":3,"thread_start_us":14091,"threads_started":1}
I20260807 18:13:20.554785 30626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:20.530631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45198 (local address 127.18.154.3:37679)
0807 18:13:20.543494 (+ 12863us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:20.543508 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:20.543552 (+ 44us) server_negotiation.cc:407] Connection header received
0807 18:13:20.543950 (+ 398us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:20.543961 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:20.544301 (+ 340us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:20.544530 (+ 229us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:13:20.545670 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:20.547364 (+ 1694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:20.550598 (+ 3234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:20.552100 (+ 1502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:20.552281 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:20.552512 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0807 18:13:20.554365 (+ 1853us) server_negotiation.cc:299] Negotiation successful
0807 18:13:20.554493 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12631,"thread_start_us":12926,"threads_started":1}
I20260807 18:13:20.555804 30622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:20.530075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.2:36977 (local address 127.0.0.1:58554)
0807 18:13:20.544318 (+ 14243us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:20.544336 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 18:13:20.544442 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:13:20.548172 (+ 3730us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:13:20.548177 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:13:20.548206 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:13:20.548820 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:20.548835 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:20.550737 (+ 1902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:20.550743 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:20.552110 (+ 1367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:20.552122 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:20.555273 (+ 3151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:20.555282 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:20.555400 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:20.555422 (+ 22us) client_negotiation.cc:769] Sending connection context
0807 18:13:20.555474 (+ 52us) client_negotiation.cc:240] Negotiation successful
0807 18:13:20.555554 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14119,"mutex_wait_us":17,"thread_start_us":14439,"threads_started":1}
I20260807 18:13:20.556788 30623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:20.530424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58554 (local address 127.18.154.2:36977)
0807 18:13:20.547568 (+ 17144us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:20.547578 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:20.547636 (+ 58us) server_negotiation.cc:407] Connection header received
0807 18:13:20.547749 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:20.547756 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:20.547956 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:20.548153 (+ 197us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:13:20.549021 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:20.550594 (+ 1573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:20.552278 (+ 1684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:20.555000 (+ 2722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:20.556051 (+ 1051us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:20.556250 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0807 18:13:20.556327 (+ 77us) server_negotiation.cc:299] Negotiation successful
0807 18:13:20.556477 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16984,"thread_start_us":16818,"threads_started":1}
I20260807 18:13:20.558950 30057 tablet_service.cc:1467] Processing CreateTablet for tablet 696e9fc6729d4a9b8c24e762912aff24 (DEFAULT_TABLE table=test-table [id=4995ce3e69014b7faa988ea9d25b07b3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:13:20.559535 30482 tablet_service.cc:1467] Processing CreateTablet for tablet 696e9fc6729d4a9b8c24e762912aff24 (DEFAULT_TABLE table=test-table [id=4995ce3e69014b7faa988ea9d25b07b3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:13:20.560165 30271 tablet_service.cc:1467] Processing CreateTablet for tablet 696e9fc6729d4a9b8c24e762912aff24 (DEFAULT_TABLE table=test-table [id=4995ce3e69014b7faa988ea9d25b07b3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:13:20.560773 30057 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 696e9fc6729d4a9b8c24e762912aff24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:13:20.560964 30482 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 696e9fc6729d4a9b8c24e762912aff24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:13:20.561470 30271 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 696e9fc6729d4a9b8c24e762912aff24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:13:20.572662 30631 tablet_bootstrap.cc:492] T 696e9fc6729d4a9b8c24e762912aff24 P a7039556d1bb4f9991853edeb64b2a62: Bootstrap starting.
I20260807 18:13:20.575563 30633 tablet_bootstrap.cc:492] T 696e9fc6729d4a9b8c24e762912aff24 P f364d0e1a3f14b55829f9ee56ac06792: Bootstrap starting.
I20260807 18:13:20.576390 30632 tablet_bootstrap.cc:492] T 696e9fc6729d4a9b8c24e762912aff24 P dd13563f5c834551b7f1830266335364: Bootstrap starting.
I20260807 18:13:20.577721 30631 tablet_bootstrap.cc:654] T 696e9fc6729d4a9b8c24e762912aff24 P a7039556d1bb4f9991853edeb64b2a62: Neither blocks nor log segments found. Creating new log.
I20260807 18:13:20.579107 30631 log.cc:824] T 696e9fc6729d4a9b8c24e762912aff24 P a7039556d1bb4f9991853edeb64b2a62: Log is configured to *not* fsync() on all Append() calls
I20260807 18:13:20.580401 30632 tablet_bootstrap.cc:654] T 696e9fc6729d4a9b8c24e762912aff24 P dd13563f5c834551b7f1830266335364: Neither blocks nor log segments found. Creating new log.
I20260807 18:13:20.581744 30632 log.cc:824] T 696e9fc6729d4a9b8c24e762912aff24 P dd13563f5c834551b7f1830266335364: Log is configured to *not* fsync() on all Append() calls
I20260807 18:13:20.582369 30631 tablet_bootstrap.cc:492] T 696e9fc6729d4a9b8c24e762912aff24 P a7039556d1bb4f9991853edeb64b2a62: No bootstrap required, opened a new log
I20260807 18:13:20.582742 30631 ts_tablet_manager.cc:1397] T 696e9fc6729d4a9b8c24e762912aff24 P a7039556d1bb4f9991853edeb64b2a62: Time spent bootstrapping tablet: real 0.010s user 0.009s sys 0.000s
I20260807 18:13:20.583473 30633 tablet_bootstrap.cc:654] T 696e9fc6729d4a9b8c24e762912aff24 P f364d0e1a3f14b55829f9ee56ac06792: Neither blocks nor log segments found. Creating new log.
I20260807 18:13:20.585083 30633 log.cc:824] T 696e9fc6729d4a9b8c24e762912aff24 P f364d0e1a3f14b55829f9ee56ac06792: Log is configured to *not* fsync() on all Append() calls
I20260807 18:13:20.588721 30632 tablet_bootstrap.cc:492] T 696e9fc6729d4a9b8c24e762912aff24 P dd13563f5c834551b7f1830266335364: No bootstrap required, opened a new log
I20260807 18:13:20.588961 30633 tablet_bootstrap.cc:492] T 696e9fc6729d4a9b8c24e762912aff24 P f364d0e1a3f14b55829f9ee56ac06792: No bootstrap required, opened a new log
I20260807 18:13:20.589026 30632 ts_tablet_manager.cc:1397] T 696e9fc6729d4a9b8c24e762912aff24 P dd13563f5c834551b7f1830266335364: Time spent bootstrapping tablet: real 0.013s user 0.006s sys 0.003s
I20260807 18:13:20.589252 30633 ts_tablet_manager.cc:1397] T 696e9fc6729d4a9b8c24e762912aff24 P f364d0e1a3f14b55829f9ee56ac06792: Time spent bootstrapping tablet: real 0.014s user 0.003s sys 0.003s
I20260807 18:13:20.591920 30631 raft_consensus.cc:348] T 696e9fc6729d4a9b8c24e762912aff24 P a7039556d1bb4f9991853edeb64b2a62 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7039556d1bb4f9991853edeb64b2a62" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 37679 } } peers { permanent_uuid: "dd13563f5c834551b7f1830266335364" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 36977 } } peers { permanent_uuid: "f364d0e1a3f14b55829f9ee56ac06792" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 46809 } }
I20260807 18:13:20.592329 30631 raft_consensus.cc:374] T 696e9fc6729d4a9b8c24e762912aff24 P a7039556d1bb4f9991853edeb64b2a62 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:13:20.592442 30631 raft_consensus.cc:729] T 696e9fc6729d4a9b8c24e762912aff24 P a7039556d1bb4f9991853edeb64b2a62 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a7039556d1bb4f9991853edeb64b2a62, State: Initialized, Role: FOLLOWER
I20260807 18:13:20.593181 30631 consensus_queue.cc:260] T 696e9fc6729d4a9b8c24e762912aff24 P a7039556d1bb4f9991853edeb64b2a62 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7039556d1bb4f9991853edeb64b2a62" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 37679 } } peers { permanent_uuid: "dd13563f5c834551b7f1830266335364" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 36977 } } peers { permanent_uuid: "f364d0e1a3f14b55829f9ee56ac06792" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 46809 } }
I20260807 18:13:20.600311 30632 raft_consensus.cc:348] T 696e9fc6729d4a9b8c24e762912aff24 P dd13563f5c834551b7f1830266335364 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7039556d1bb4f9991853edeb64b2a62" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 37679 } } peers { permanent_uuid: "dd13563f5c834551b7f1830266335364" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 36977 } } peers { permanent_uuid: "f364d0e1a3f14b55829f9ee56ac06792" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 46809 } }
I20260807 18:13:20.600827 30632 raft_consensus.cc:374] T 696e9fc6729d4a9b8c24e762912aff24 P dd13563f5c834551b7f1830266335364 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:13:20.600946 30632 raft_consensus.cc:729] T 696e9fc6729d4a9b8c24e762912aff24 P dd13563f5c834551b7f1830266335364 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd13563f5c834551b7f1830266335364, State: Initialized, Role: FOLLOWER
I20260807 18:13:20.601604 30632 consensus_queue.cc:260] T 696e9fc6729d4a9b8c24e762912aff24 P dd13563f5c834551b7f1830266335364 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7039556d1bb4f9991853edeb64b2a62" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 37679 } } peers { permanent_uuid: "dd13563f5c834551b7f1830266335364" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 36977 } } peers { permanent_uuid: "f364d0e1a3f14b55829f9ee56ac06792" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 46809 } }
I20260807 18:13:20.603519 30632 ts_tablet_manager.cc:1428] T 696e9fc6729d4a9b8c24e762912aff24 P dd13563f5c834551b7f1830266335364: Time spent starting tablet: real 0.014s user 0.013s sys 0.000s
I20260807 18:13:20.604250 30633 raft_consensus.cc:348] T 696e9fc6729d4a9b8c24e762912aff24 P f364d0e1a3f14b55829f9ee56ac06792 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7039556d1bb4f9991853edeb64b2a62" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 37679 } } peers { permanent_uuid: "dd13563f5c834551b7f1830266335364" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 36977 } } peers { permanent_uuid: "f364d0e1a3f14b55829f9ee56ac06792" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 46809 } }
I20260807 18:13:20.604703 30633 raft_consensus.cc:374] T 696e9fc6729d4a9b8c24e762912aff24 P f364d0e1a3f14b55829f9ee56ac06792 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:13:20.604828 30633 raft_consensus.cc:729] T 696e9fc6729d4a9b8c24e762912aff24 P f364d0e1a3f14b55829f9ee56ac06792 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f364d0e1a3f14b55829f9ee56ac06792, State: Initialized, Role: FOLLOWER
I20260807 18:13:20.605162 30640 raft_consensus.cc:482] T 696e9fc6729d4a9b8c24e762912aff24 P a7039556d1bb4f9991853edeb64b2a62 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:13:20.605561 30640 raft_consensus.cc:504] T 696e9fc6729d4a9b8c24e762912aff24 P a7039556d1bb4f9991853edeb64b2a62 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7039556d1bb4f9991853edeb64b2a62" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 37679 } } peers { permanent_uuid: "dd13563f5c834551b7f1830266335364" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 36977 } } peers { permanent_uuid: "f364d0e1a3f14b55829f9ee56ac06792" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 46809 } }
I20260807 18:13:20.606947 30633 consensus_queue.cc:260] T 696e9fc6729d4a9b8c24e762912aff24 P f364d0e1a3f14b55829f9ee56ac06792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7039556d1bb4f9991853edeb64b2a62" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 37679 } } peers { permanent_uuid: "dd13563f5c834551b7f1830266335364" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 36977 } } peers { permanent_uuid: "f364d0e1a3f14b55829f9ee56ac06792" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 46809 } }
I20260807 18:13:20.609177 30631 ts_tablet_manager.cc:1428] T 696e9fc6729d4a9b8c24e762912aff24 P a7039556d1bb4f9991853edeb64b2a62: Time spent starting tablet: real 0.026s user 0.009s sys 0.018s
I20260807 18:13:20.611002 30633 ts_tablet_manager.cc:1428] T 696e9fc6729d4a9b8c24e762912aff24 P f364d0e1a3f14b55829f9ee56ac06792: Time spent starting tablet: real 0.022s user 0.009s sys 0.006s
I20260807 18:13:20.611610 30640 leader_election.cc:288] T 696e9fc6729d4a9b8c24e762912aff24 P a7039556d1bb4f9991853edeb64b2a62 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dd13563f5c834551b7f1830266335364 (127.18.154.2:36977), f364d0e1a3f14b55829f9ee56ac06792 (127.18.154.1:46809)
I20260807 18:13:20.618884 30629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:20.612426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.3:55631 (local address 127.18.154.1:46809)
0807 18:13:20.612608 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:20.612616 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:20.613093 (+ 477us) server_negotiation.cc:407] Connection header received
0807 18:13:20.613278 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:20.613284 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:20.613392 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:20.613503 (+ 111us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:13:20.614229 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:20.614925 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:20.616977 (+ 2052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:20.617872 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:20.618031 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:20.618107 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0807 18:13:20.618483 (+ 376us) server_negotiation.cc:299] Negotiation successful
0807 18:13:20.618629 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42,"spinlock_wait_cycles":4608}
I20260807 18:13:20.618991 30645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:20.611863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.1:46809 (local address 127.18.154.3:55631)
0807 18:13:20.612982 (+ 1119us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:20.613001 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 18:13:20.613138 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:13:20.613574 (+ 436us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:13:20.613579 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:13:20.613603 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:13:20.614029 (+ 426us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:20.614044 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:20.615075 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:20.615082 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:20.616813 (+ 1731us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:20.616826 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:20.618097 (+ 1271us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:20.618104 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:20.618293 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:20.618331 (+ 38us) client_negotiation.cc:769] Sending connection context
0807 18:13:20.618441 (+ 110us) client_negotiation.cc:240] Negotiation successful
0807 18:13:20.618597 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":966,"thread_start_us":840,"threads_started":1}
I20260807 18:13:20.620306 30623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:20.608503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.3:34151 (local address 127.18.154.2:36977)
0807 18:13:20.608727 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:20.608733 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:20.612451 (+ 3718us) server_negotiation.cc:407] Connection header received
0807 18:13:20.612762 (+ 311us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:20.612770 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:20.612862 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:20.612926 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:13:20.613787 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:20.615026 (+ 1239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:20.617882 (+ 2856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:20.619074 (+ 1192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:20.619241 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:20.619317 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0807 18:13:20.619904 (+ 587us) server_negotiation.cc:299] Negotiation successful
0807 18:13:20.620060 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260807 18:13:20.620309 30643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:20.608006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.2:36977 (local address 127.18.154.3:34151)
0807 18:13:20.612338 (+ 4332us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:20.612360 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 18:13:20.612562 (+ 202us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:13:20.613142 (+ 580us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:13:20.613146 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:13:20.613170 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:13:20.613574 (+ 404us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:20.613604 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:20.615206 (+ 1602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:20.615218 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:20.617703 (+ 2485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:20.617720 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:20.619504 (+ 1784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:20.619512 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:20.619694 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:20.619719 (+ 25us) client_negotiation.cc:769] Sending connection context
0807 18:13:20.619822 (+ 103us) client_negotiation.cc:240] Negotiation successful
0807 18:13:20.619974 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4199,"thread_start_us":4083,"threads_started":1}
I20260807 18:13:20.620312 30093 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "696e9fc6729d4a9b8c24e762912aff24" candidate_uuid: "a7039556d1bb4f9991853edeb64b2a62" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f364d0e1a3f14b55829f9ee56ac06792" is_pre_election: true
I20260807 18:13:20.620859 30093 raft_consensus.cc:2454] T 696e9fc6729d4a9b8c24e762912aff24 P f364d0e1a3f14b55829f9ee56ac06792 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a7039556d1bb4f9991853edeb64b2a62 in term 0.
W20260807 18:13:20.621181 30377 tablet.cc:2369] T 696e9fc6729d4a9b8c24e762912aff24 P dd13563f5c834551b7f1830266335364: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:13:20.621500 30300 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "696e9fc6729d4a9b8c24e762912aff24" candidate_uuid: "a7039556d1bb4f9991853edeb64b2a62" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd13563f5c834551b7f1830266335364" is_pre_election: true
I20260807 18:13:20.621815 30406 leader_election.cc:302] T 696e9fc6729d4a9b8c24e762912aff24 P a7039556d1bb4f9991853edeb64b2a62 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a7039556d1bb4f9991853edeb64b2a62, f364d0e1a3f14b55829f9ee56ac06792; no voters:
I20260807 18:13:20.622171 30300 raft_consensus.cc:2454] T 696e9fc6729d4a9b8c24e762912aff24 P dd13563f5c834551b7f1830266335364 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a7039556d1bb4f9991853edeb64b2a62 in term 0.
I20260807 18:13:20.622262 30640 raft_consensus.cc:2781] T 696e9fc6729d4a9b8c24e762912aff24 P a7039556d1bb4f9991853edeb64b2a62 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:13:20.622498 30640 raft_consensus.cc:482] T 696e9fc6729d4a9b8c24e762912aff24 P a7039556d1bb4f9991853edeb64b2a62 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:13:20.622618 30640 raft_consensus.cc:3037] T 696e9fc6729d4a9b8c24e762912aff24 P a7039556d1bb4f9991853edeb64b2a62 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:13:20.626138 30640 raft_consensus.cc:504] T 696e9fc6729d4a9b8c24e762912aff24 P a7039556d1bb4f9991853edeb64b2a62 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7039556d1bb4f9991853edeb64b2a62" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 37679 } } peers { permanent_uuid: "dd13563f5c834551b7f1830266335364" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 36977 } } peers { permanent_uuid: "f364d0e1a3f14b55829f9ee56ac06792" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 46809 } }
I20260807 18:13:20.626942 30640 leader_election.cc:288] T 696e9fc6729d4a9b8c24e762912aff24 P a7039556d1bb4f9991853edeb64b2a62 [CANDIDATE]: Term 1 election: Requested vote from peers dd13563f5c834551b7f1830266335364 (127.18.154.2:36977), f364d0e1a3f14b55829f9ee56ac06792 (127.18.154.1:46809)
I20260807 18:13:20.627339 30300 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "696e9fc6729d4a9b8c24e762912aff24" candidate_uuid: "a7039556d1bb4f9991853edeb64b2a62" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd13563f5c834551b7f1830266335364"
I20260807 18:13:20.627453 30093 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "696e9fc6729d4a9b8c24e762912aff24" candidate_uuid: "a7039556d1bb4f9991853edeb64b2a62" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f364d0e1a3f14b55829f9ee56ac06792"
I20260807 18:13:20.627559 30300 raft_consensus.cc:3037] T 696e9fc6729d4a9b8c24e762912aff24 P dd13563f5c834551b7f1830266335364 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:13:20.627717 30093 raft_consensus.cc:3037] T 696e9fc6729d4a9b8c24e762912aff24 P f364d0e1a3f14b55829f9ee56ac06792 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:13:20.630373 30093 raft_consensus.cc:2454] T 696e9fc6729d4a9b8c24e762912aff24 P f364d0e1a3f14b55829f9ee56ac06792 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a7039556d1bb4f9991853edeb64b2a62 in term 1.
I20260807 18:13:20.630775 30406 leader_election.cc:302] T 696e9fc6729d4a9b8c24e762912aff24 P a7039556d1bb4f9991853edeb64b2a62 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a7039556d1bb4f9991853edeb64b2a62, f364d0e1a3f14b55829f9ee56ac06792; no voters:
I20260807 18:13:20.631062 30640 raft_consensus.cc:2781] T 696e9fc6729d4a9b8c24e762912aff24 P a7039556d1bb4f9991853edeb64b2a62 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:13:20.631189 30300 raft_consensus.cc:2454] T 696e9fc6729d4a9b8c24e762912aff24 P dd13563f5c834551b7f1830266335364 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a7039556d1bb4f9991853edeb64b2a62 in term 1.
I20260807 18:13:20.631248 30640 raft_consensus.cc:686] T 696e9fc6729d4a9b8c24e762912aff24 P a7039556d1bb4f9991853edeb64b2a62 [term 1 LEADER]: Becoming Leader. State: Replica: a7039556d1bb4f9991853edeb64b2a62, State: Running, Role: LEADER
I20260807 18:13:20.631843 30640 consensus_queue.cc:237] T 696e9fc6729d4a9b8c24e762912aff24 P a7039556d1bb4f9991853edeb64b2a62 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7039556d1bb4f9991853edeb64b2a62" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 37679 } } peers { permanent_uuid: "dd13563f5c834551b7f1830266335364" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 36977 } } peers { permanent_uuid: "f364d0e1a3f14b55829f9ee56ac06792" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 46809 } }
I20260807 18:13:20.636461 29847 catalog_manager.cc:5495] T 696e9fc6729d4a9b8c24e762912aff24 P a7039556d1bb4f9991853edeb64b2a62 reported cstate change: term changed from 0 to 1, leader changed from <none> to a7039556d1bb4f9991853edeb64b2a62 (127.18.154.3). New cstate: current_term: 1 leader_uuid: "a7039556d1bb4f9991853edeb64b2a62" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7039556d1bb4f9991853edeb64b2a62" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 37679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd13563f5c834551b7f1830266335364" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 36977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f364d0e1a3f14b55829f9ee56ac06792" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 46809 } health_report { overall_health: UNKNOWN } } }
I20260807 18:13:20.656415 30626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:20.650884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45206 (local address 127.18.154.3:37679)
0807 18:13:20.651112 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:20.651121 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:20.651205 (+ 84us) server_negotiation.cc:407] Connection header received
0807 18:13:20.651473 (+ 268us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:20.651477 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:20.651554 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:20.651713 (+ 159us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:13:20.652083 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:20.652839 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:20.654248 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:20.654670 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:20.654810 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:20.655274 (+ 464us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:13:20.655811 (+ 537us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:13:20.655983 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0807 18:13:20.656043 (+ 60us) server_negotiation.cc:299] Negotiation successful
0807 18:13:20.656141 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260807 18:13:20.665393 30093 raft_consensus.cc:1261] T 696e9fc6729d4a9b8c24e762912aff24 P f364d0e1a3f14b55829f9ee56ac06792 [term 1 FOLLOWER]: Refusing update from remote peer a7039556d1bb4f9991853edeb64b2a62: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:13:20.665439 30300 raft_consensus.cc:1261] T 696e9fc6729d4a9b8c24e762912aff24 P dd13563f5c834551b7f1830266335364 [term 1 FOLLOWER]: Refusing update from remote peer a7039556d1bb4f9991853edeb64b2a62: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:13:20.666494 30639 consensus_queue.cc:1035] T 696e9fc6729d4a9b8c24e762912aff24 P a7039556d1bb4f9991853edeb64b2a62 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f364d0e1a3f14b55829f9ee56ac06792" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 46809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:13:20.666880 30640 consensus_queue.cc:1035] T 696e9fc6729d4a9b8c24e762912aff24 P a7039556d1bb4f9991853edeb64b2a62 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd13563f5c834551b7f1830266335364" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 36977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:13:20.685856 30649 mvcc.cc:205] Tried to move back new op lower bound from 7315973737115308032 to 7315973736992059392. Current Snapshot: MvccSnapshot[applied={T|T < 7315973737115308032 or (T in {7315973737115308032})}]
I20260807 18:13:20.695238 29848 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50446:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:13:20.695551 29848 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50446:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:13:20.699254 29848 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e61bfa7995fe4afcbcaf0939bac87929: Sending DeleteTablet for 3 replicas of tablet 696e9fc6729d4a9b8c24e762912aff24
I20260807 18:13:20.700466 30482 tablet_service.cc:1514] Processing DeleteTablet for tablet 696e9fc6729d4a9b8c24e762912aff24 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:13:20 UTC) from {username='oryx', principal='oryx/127.18.154.62@KRBTEST.COM'} at 127.0.0.1:45198
I20260807 18:13:20.700573 30271 tablet_service.cc:1514] Processing DeleteTablet for tablet 696e9fc6729d4a9b8c24e762912aff24 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:13:20 UTC) from {username='oryx', principal='oryx/127.18.154.62@KRBTEST.COM'} at 127.0.0.1:58554
I20260807 18:13:20.700649 30057 tablet_service.cc:1514] Processing DeleteTablet for tablet 696e9fc6729d4a9b8c24e762912aff24 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:13:20 UTC) from {username='oryx', principal='oryx/127.18.154.62@KRBTEST.COM'} at 127.0.0.1:60300
I20260807 18:13:20.705575 29848 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50446:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 18:13:20.706809 29848 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:13:20.711423 30668 tablet_replica.cc:330] T 696e9fc6729d4a9b8c24e762912aff24 P a7039556d1bb4f9991853edeb64b2a62: stopping tablet replica
I20260807 18:13:20.712213 30668 raft_consensus.cc:2229] T 696e9fc6729d4a9b8c24e762912aff24 P a7039556d1bb4f9991853edeb64b2a62 [term 1 LEADER]: Raft consensus shutting down.
I20260807 18:13:20.712545 30671 tablet_replica.cc:330] T 696e9fc6729d4a9b8c24e762912aff24 P f364d0e1a3f14b55829f9ee56ac06792: stopping tablet replica
I20260807 18:13:20.713045 30668 raft_consensus.cc:2258] T 696e9fc6729d4a9b8c24e762912aff24 P a7039556d1bb4f9991853edeb64b2a62 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:13:20.713146 30671 raft_consensus.cc:2229] T 696e9fc6729d4a9b8c24e762912aff24 P f364d0e1a3f14b55829f9ee56ac06792 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:13:20.713956 30671 raft_consensus.cc:2258] T 696e9fc6729d4a9b8c24e762912aff24 P f364d0e1a3f14b55829f9ee56ac06792 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:13:20.714561 30669 tablet_replica.cc:330] T 696e9fc6729d4a9b8c24e762912aff24 P dd13563f5c834551b7f1830266335364: stopping tablet replica
I20260807 18:13:20.715801 30668 ts_tablet_manager.cc:1905] T 696e9fc6729d4a9b8c24e762912aff24 P a7039556d1bb4f9991853edeb64b2a62: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:13:20.716287 30669 raft_consensus.cc:2229] T 696e9fc6729d4a9b8c24e762912aff24 P dd13563f5c834551b7f1830266335364 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:13:20.717329 30671 ts_tablet_manager.cc:1905] T 696e9fc6729d4a9b8c24e762912aff24 P f364d0e1a3f14b55829f9ee56ac06792: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:13:20.718021 30669 raft_consensus.cc:2258] T 696e9fc6729d4a9b8c24e762912aff24 P dd13563f5c834551b7f1830266335364 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:13:20.718619 30482 tablet_service.cc:1467] Processing CreateTablet for tablet 4ca8f5095320465d89d9dad5e011a234 (DEFAULT_TABLE table=test-table [id=4da3f1990d6f4bb79796a2d5fbe62e37]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:13:20.719121 30482 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4ca8f5095320465d89d9dad5e011a234. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:13:20.720733 30057 tablet_service.cc:1467] Processing CreateTablet for tablet 4ca8f5095320465d89d9dad5e011a234 (DEFAULT_TABLE table=test-table [id=4da3f1990d6f4bb79796a2d5fbe62e37]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:13:20.721283 30057 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4ca8f5095320465d89d9dad5e011a234. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:13:20.722203 30271 tablet_service.cc:1467] Processing CreateTablet for tablet 4ca8f5095320465d89d9dad5e011a234 (DEFAULT_TABLE table=test-table [id=4da3f1990d6f4bb79796a2d5fbe62e37]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:13:20.723860 30668 ts_tablet_manager.cc:1918] T 696e9fc6729d4a9b8c24e762912aff24 P a7039556d1bb4f9991853edeb64b2a62: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:13:20.724081 30668 log.cc:1196] T 696e9fc6729d4a9b8c24e762912aff24 P a7039556d1bb4f9991853edeb64b2a62: Deleting WAL directory at /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126328808105-19048-0/minicluster-data/ts-2/wal/wals/696e9fc6729d4a9b8c24e762912aff24
I20260807 18:13:20.724682 30668 ts_tablet_manager.cc:1939] T 696e9fc6729d4a9b8c24e762912aff24 P a7039556d1bb4f9991853edeb64b2a62: Deleting consensus metadata
I20260807 18:13:20.725847 30271 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4ca8f5095320465d89d9dad5e011a234. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:13:20.728034 30633 tablet_bootstrap.cc:492] T 4ca8f5095320465d89d9dad5e011a234 P f364d0e1a3f14b55829f9ee56ac06792: Bootstrap starting.
I20260807 18:13:20.728729 29823 catalog_manager.cc:4841] TS a7039556d1bb4f9991853edeb64b2a62 (127.18.154.3:37679): tablet 696e9fc6729d4a9b8c24e762912aff24 (table test-table [id=4995ce3e69014b7faa988ea9d25b07b3]) successfully deleted
I20260807 18:13:20.729817 30633 tablet_bootstrap.cc:654] T 4ca8f5095320465d89d9dad5e011a234 P f364d0e1a3f14b55829f9ee56ac06792: Neither blocks nor log segments found. Creating new log.
I20260807 18:13:20.730232 30631 tablet_bootstrap.cc:492] T 4ca8f5095320465d89d9dad5e011a234 P a7039556d1bb4f9991853edeb64b2a62: Bootstrap starting.
I20260807 18:13:20.731276 30669 ts_tablet_manager.cc:1905] T 696e9fc6729d4a9b8c24e762912aff24 P dd13563f5c834551b7f1830266335364: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:13:20.731968 30631 tablet_bootstrap.cc:654] T 4ca8f5095320465d89d9dad5e011a234 P a7039556d1bb4f9991853edeb64b2a62: Neither blocks nor log segments found. Creating new log.
I20260807 18:13:20.737828 30632 tablet_bootstrap.cc:492] T 4ca8f5095320465d89d9dad5e011a234 P dd13563f5c834551b7f1830266335364: Bootstrap starting.
I20260807 18:13:20.741158 30632 tablet_bootstrap.cc:654] T 4ca8f5095320465d89d9dad5e011a234 P dd13563f5c834551b7f1830266335364: Neither blocks nor log segments found. Creating new log.
I20260807 18:13:20.741595 30633 tablet_bootstrap.cc:492] T 4ca8f5095320465d89d9dad5e011a234 P f364d0e1a3f14b55829f9ee56ac06792: No bootstrap required, opened a new log
I20260807 18:13:20.741770 30633 ts_tablet_manager.cc:1397] T 4ca8f5095320465d89d9dad5e011a234 P f364d0e1a3f14b55829f9ee56ac06792: Time spent bootstrapping tablet: real 0.014s user 0.003s sys 0.009s
I20260807 18:13:20.742726 30633 raft_consensus.cc:348] T 4ca8f5095320465d89d9dad5e011a234 P f364d0e1a3f14b55829f9ee56ac06792 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f364d0e1a3f14b55829f9ee56ac06792" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 46809 } } peers { permanent_uuid: "dd13563f5c834551b7f1830266335364" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 36977 } } peers { permanent_uuid: "a7039556d1bb4f9991853edeb64b2a62" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 37679 } }
I20260807 18:13:20.742986 30633 raft_consensus.cc:374] T 4ca8f5095320465d89d9dad5e011a234 P f364d0e1a3f14b55829f9ee56ac06792 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:13:20.743078 30633 raft_consensus.cc:729] T 4ca8f5095320465d89d9dad5e011a234 P f364d0e1a3f14b55829f9ee56ac06792 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f364d0e1a3f14b55829f9ee56ac06792, State: Initialized, Role: FOLLOWER
I20260807 18:13:20.743341 30633 consensus_queue.cc:260] T 4ca8f5095320465d89d9dad5e011a234 P f364d0e1a3f14b55829f9ee56ac06792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f364d0e1a3f14b55829f9ee56ac06792" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 46809 } } peers { permanent_uuid: "dd13563f5c834551b7f1830266335364" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 36977 } } peers { permanent_uuid: "a7039556d1bb4f9991853edeb64b2a62" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 37679 } }
I20260807 18:13:20.744040 30671 ts_tablet_manager.cc:1918] T 696e9fc6729d4a9b8c24e762912aff24 P f364d0e1a3f14b55829f9ee56ac06792: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:13:20.745340 30631 tablet_bootstrap.cc:492] T 4ca8f5095320465d89d9dad5e011a234 P a7039556d1bb4f9991853edeb64b2a62: No bootstrap required, opened a new log
I20260807 18:13:20.745544 30631 ts_tablet_manager.cc:1397] T 4ca8f5095320465d89d9dad5e011a234 P a7039556d1bb4f9991853edeb64b2a62: Time spent bootstrapping tablet: real 0.015s user 0.006s sys 0.006s
I20260807 18:13:20.744208 30633 ts_tablet_manager.cc:1428] T 4ca8f5095320465d89d9dad5e011a234 P f364d0e1a3f14b55829f9ee56ac06792: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260807 18:13:20.745587 30671 log.cc:1196] T 696e9fc6729d4a9b8c24e762912aff24 P f364d0e1a3f14b55829f9ee56ac06792: Deleting WAL directory at /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126328808105-19048-0/minicluster-data/ts-0/wal/wals/696e9fc6729d4a9b8c24e762912aff24
I20260807 18:13:20.745996 30632 tablet_bootstrap.cc:492] T 4ca8f5095320465d89d9dad5e011a234 P dd13563f5c834551b7f1830266335364: No bootstrap required, opened a new log
I20260807 18:13:20.746219 30632 ts_tablet_manager.cc:1397] T 4ca8f5095320465d89d9dad5e011a234 P dd13563f5c834551b7f1830266335364: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.004s
I20260807 18:13:20.746474 30671 ts_tablet_manager.cc:1939] T 696e9fc6729d4a9b8c24e762912aff24 P f364d0e1a3f14b55829f9ee56ac06792: Deleting consensus metadata
I20260807 18:13:20.746474 30631 raft_consensus.cc:348] T 4ca8f5095320465d89d9dad5e011a234 P a7039556d1bb4f9991853edeb64b2a62 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f364d0e1a3f14b55829f9ee56ac06792" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 46809 } } peers { permanent_uuid: "dd13563f5c834551b7f1830266335364" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 36977 } } peers { permanent_uuid: "a7039556d1bb4f9991853edeb64b2a62" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 37679 } }
I20260807 18:13:20.746709 30631 raft_consensus.cc:374] T 4ca8f5095320465d89d9dad5e011a234 P a7039556d1bb4f9991853edeb64b2a62 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:13:20.746810 30631 raft_consensus.cc:729] T 4ca8f5095320465d89d9dad5e011a234 P a7039556d1bb4f9991853edeb64b2a62 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a7039556d1bb4f9991853edeb64b2a62, State: Initialized, Role: FOLLOWER
I20260807 18:13:20.747114 30631 consensus_queue.cc:260] T 4ca8f5095320465d89d9dad5e011a234 P a7039556d1bb4f9991853edeb64b2a62 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f364d0e1a3f14b55829f9ee56ac06792" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 46809 } } peers { permanent_uuid: "dd13563f5c834551b7f1830266335364" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 36977 } } peers { permanent_uuid: "a7039556d1bb4f9991853edeb64b2a62" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 37679 } }
I20260807 18:13:20.747149 30632 raft_consensus.cc:348] T 4ca8f5095320465d89d9dad5e011a234 P dd13563f5c834551b7f1830266335364 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f364d0e1a3f14b55829f9ee56ac06792" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 46809 } } peers { permanent_uuid: "dd13563f5c834551b7f1830266335364" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 36977 } } peers { permanent_uuid: "a7039556d1bb4f9991853edeb64b2a62" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 37679 } }
I20260807 18:13:20.747507 30632 raft_consensus.cc:374] T 4ca8f5095320465d89d9dad5e011a234 P dd13563f5c834551b7f1830266335364 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:13:20.747700 30632 raft_consensus.cc:729] T 4ca8f5095320465d89d9dad5e011a234 P dd13563f5c834551b7f1830266335364 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd13563f5c834551b7f1830266335364, State: Initialized, Role: FOLLOWER
I20260807 18:13:20.747732 30631 ts_tablet_manager.cc:1428] T 4ca8f5095320465d89d9dad5e011a234 P a7039556d1bb4f9991853edeb64b2a62: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260807 18:13:20.748080 30632 consensus_queue.cc:260] T 4ca8f5095320465d89d9dad5e011a234 P dd13563f5c834551b7f1830266335364 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f364d0e1a3f14b55829f9ee56ac06792" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 46809 } } peers { permanent_uuid: "dd13563f5c834551b7f1830266335364" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 36977 } } peers { permanent_uuid: "a7039556d1bb4f9991853edeb64b2a62" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 37679 } }
I20260807 18:13:20.749455 30632 ts_tablet_manager.cc:1428] T 4ca8f5095320465d89d9dad5e011a234 P dd13563f5c834551b7f1830266335364: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260807 18:13:20.750087 30669 ts_tablet_manager.cc:1918] T 696e9fc6729d4a9b8c24e762912aff24 P dd13563f5c834551b7f1830266335364: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:13:20.750303 30669 log.cc:1196] T 696e9fc6729d4a9b8c24e762912aff24 P dd13563f5c834551b7f1830266335364: Deleting WAL directory at /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126328808105-19048-0/minicluster-data/ts-1/wal/wals/696e9fc6729d4a9b8c24e762912aff24
I20260807 18:13:20.750275 29821 catalog_manager.cc:4841] TS f364d0e1a3f14b55829f9ee56ac06792 (127.18.154.1:46809): tablet 696e9fc6729d4a9b8c24e762912aff24 (table test-table [id=4995ce3e69014b7faa988ea9d25b07b3]) successfully deleted
I20260807 18:13:20.750968 30669 ts_tablet_manager.cc:1939] T 696e9fc6729d4a9b8c24e762912aff24 P dd13563f5c834551b7f1830266335364: Deleting consensus metadata
I20260807 18:13:20.753044 29819 catalog_manager.cc:4841] TS dd13563f5c834551b7f1830266335364 (127.18.154.2:36977): tablet 696e9fc6729d4a9b8c24e762912aff24 (table test-table [id=4995ce3e69014b7faa988ea9d25b07b3]) successfully deleted
W20260807 18:13:20.816757 30161 tablet.cc:2369] T 4ca8f5095320465d89d9dad5e011a234 P f364d0e1a3f14b55829f9ee56ac06792: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 18:13:20.956869 30589 tablet.cc:2369] T 4ca8f5095320465d89d9dad5e011a234 P a7039556d1bb4f9991853edeb64b2a62: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:13:21.014576 30641 raft_consensus.cc:482] T 4ca8f5095320465d89d9dad5e011a234 P dd13563f5c834551b7f1830266335364 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:13:21.014855 30641 raft_consensus.cc:504] T 4ca8f5095320465d89d9dad5e011a234 P dd13563f5c834551b7f1830266335364 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f364d0e1a3f14b55829f9ee56ac06792" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 46809 } } peers { permanent_uuid: "dd13563f5c834551b7f1830266335364" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 36977 } } peers { permanent_uuid: "a7039556d1bb4f9991853edeb64b2a62" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 37679 } }
I20260807 18:13:21.016701 30641 leader_election.cc:288] T 4ca8f5095320465d89d9dad5e011a234 P dd13563f5c834551b7f1830266335364 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f364d0e1a3f14b55829f9ee56ac06792 (127.18.154.1:46809), a7039556d1bb4f9991853edeb64b2a62 (127.18.154.3:37679)
I20260807 18:13:21.025210 30626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:21.017481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.2:45141 (local address 127.18.154.3:37679)
0807 18:13:21.017720 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:21.017727 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:21.017962 (+ 235us) server_negotiation.cc:407] Connection header received
0807 18:13:21.018158 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:21.018165 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:21.018253 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:21.018324 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:13:21.019174 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:21.020488 (+ 1314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:21.022775 (+ 2287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:21.024033 (+ 1258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:21.024215 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:21.024305 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0807 18:13:21.024728 (+ 423us) server_negotiation.cc:299] Negotiation successful
0807 18:13:21.024880 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
I20260807 18:13:21.025624 30676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:21.016910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.1:46809 (local address 127.18.154.2:53153)
0807 18:13:21.017613 (+ 703us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:21.017637 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 18:13:21.017811 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:13:21.018501 (+ 690us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:13:21.018507 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:13:21.018542 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:13:21.019148 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:21.019164 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:21.020611 (+ 1447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:21.020617 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:21.023093 (+ 2476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:21.023110 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:21.024737 (+ 1627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:21.024743 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:21.024914 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:21.024937 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 18:13:21.025042 (+ 105us) client_negotiation.cc:240] Negotiation successful
0807 18:13:21.025177 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":494,"mutex_wait_us":75,"thread_start_us":334,"threads_started":1}
I20260807 18:13:21.025883 30629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:21.017903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.2:53153 (local address 127.18.154.1:46809)
0807 18:13:21.018160 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:21.018166 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:21.018182 (+ 16us) server_negotiation.cc:407] Connection header received
0807 18:13:21.018251 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:21.018255 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:21.018348 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:21.018441 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:13:21.019363 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:21.020462 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:21.023265 (+ 2803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:21.024515 (+ 1250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:21.024707 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:21.024796 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0807 18:13:21.025080 (+ 284us) server_negotiation.cc:299] Negotiation successful
0807 18:13:21.025215 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
I20260807 18:13:21.026111 30677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:21.017066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.3:37679 (local address 127.18.154.2:45141)
0807 18:13:21.017874 (+ 808us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:21.017890 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 18:13:21.017999 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:13:21.018564 (+ 565us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:13:21.018568 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:13:21.018614 (+ 46us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:13:21.018993 (+ 379us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:21.019003 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:21.020659 (+ 1656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:21.020662 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:21.022607 (+ 1945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:21.022621 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:21.024283 (+ 1662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:21.024291 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:21.024476 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:21.024513 (+ 37us) client_negotiation.cc:769] Sending connection context
0807 18:13:21.025503 (+ 990us) client_negotiation.cc:240] Negotiation successful
0807 18:13:21.025710 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":693,"mutex_wait_us":29,"thread_start_us":904,"threads_started":1}
I20260807 18:13:21.026994 30093 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ca8f5095320465d89d9dad5e011a234" candidate_uuid: "dd13563f5c834551b7f1830266335364" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f364d0e1a3f14b55829f9ee56ac06792" is_pre_election: true
I20260807 18:13:21.027334 30093 raft_consensus.cc:2454] T 4ca8f5095320465d89d9dad5e011a234 P f364d0e1a3f14b55829f9ee56ac06792 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd13563f5c834551b7f1830266335364 in term 0.
I20260807 18:13:21.027141 30526 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ca8f5095320465d89d9dad5e011a234" candidate_uuid: "dd13563f5c834551b7f1830266335364" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a7039556d1bb4f9991853edeb64b2a62" is_pre_election: true
I20260807 18:13:21.027674 30526 raft_consensus.cc:2454] T 4ca8f5095320465d89d9dad5e011a234 P a7039556d1bb4f9991853edeb64b2a62 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd13563f5c834551b7f1830266335364 in term 0.
I20260807 18:13:21.027884 30197 leader_election.cc:302] T 4ca8f5095320465d89d9dad5e011a234 P dd13563f5c834551b7f1830266335364 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: dd13563f5c834551b7f1830266335364, f364d0e1a3f14b55829f9ee56ac06792; no voters:
I20260807 18:13:21.028379 30641 raft_consensus.cc:2781] T 4ca8f5095320465d89d9dad5e011a234 P dd13563f5c834551b7f1830266335364 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:13:21.028520 30641 raft_consensus.cc:482] T 4ca8f5095320465d89d9dad5e011a234 P dd13563f5c834551b7f1830266335364 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:13:21.028646 30641 raft_consensus.cc:3037] T 4ca8f5095320465d89d9dad5e011a234 P dd13563f5c834551b7f1830266335364 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:13:21.031026 30641 raft_consensus.cc:504] T 4ca8f5095320465d89d9dad5e011a234 P dd13563f5c834551b7f1830266335364 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f364d0e1a3f14b55829f9ee56ac06792" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 46809 } } peers { permanent_uuid: "dd13563f5c834551b7f1830266335364" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 36977 } } peers { permanent_uuid: "a7039556d1bb4f9991853edeb64b2a62" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 37679 } }
I20260807 18:13:21.031684 30641 leader_election.cc:288] T 4ca8f5095320465d89d9dad5e011a234 P dd13563f5c834551b7f1830266335364 [CANDIDATE]: Term 1 election: Requested vote from peers f364d0e1a3f14b55829f9ee56ac06792 (127.18.154.1:46809), a7039556d1bb4f9991853edeb64b2a62 (127.18.154.3:37679)
I20260807 18:13:21.032178 30093 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ca8f5095320465d89d9dad5e011a234" candidate_uuid: "dd13563f5c834551b7f1830266335364" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f364d0e1a3f14b55829f9ee56ac06792"
I20260807 18:13:21.032244 30526 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ca8f5095320465d89d9dad5e011a234" candidate_uuid: "dd13563f5c834551b7f1830266335364" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a7039556d1bb4f9991853edeb64b2a62"
I20260807 18:13:21.032388 30093 raft_consensus.cc:3037] T 4ca8f5095320465d89d9dad5e011a234 P f364d0e1a3f14b55829f9ee56ac06792 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:13:21.032476 30526 raft_consensus.cc:3037] T 4ca8f5095320465d89d9dad5e011a234 P a7039556d1bb4f9991853edeb64b2a62 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:13:21.034646 30093 raft_consensus.cc:2454] T 4ca8f5095320465d89d9dad5e011a234 P f364d0e1a3f14b55829f9ee56ac06792 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd13563f5c834551b7f1830266335364 in term 1.
I20260807 18:13:21.035120 30197 leader_election.cc:302] T 4ca8f5095320465d89d9dad5e011a234 P dd13563f5c834551b7f1830266335364 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: dd13563f5c834551b7f1830266335364, f364d0e1a3f14b55829f9ee56ac06792; no voters:
I20260807 18:13:21.035444 30641 raft_consensus.cc:2781] T 4ca8f5095320465d89d9dad5e011a234 P dd13563f5c834551b7f1830266335364 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:13:21.035996 30526 raft_consensus.cc:2454] T 4ca8f5095320465d89d9dad5e011a234 P a7039556d1bb4f9991853edeb64b2a62 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd13563f5c834551b7f1830266335364 in term 1.
I20260807 18:13:21.036605 30641 raft_consensus.cc:686] T 4ca8f5095320465d89d9dad5e011a234 P dd13563f5c834551b7f1830266335364 [term 1 LEADER]: Becoming Leader. State: Replica: dd13563f5c834551b7f1830266335364, State: Running, Role: LEADER
I20260807 18:13:21.037030 30641 consensus_queue.cc:237] T 4ca8f5095320465d89d9dad5e011a234 P dd13563f5c834551b7f1830266335364 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f364d0e1a3f14b55829f9ee56ac06792" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 46809 } } peers { permanent_uuid: "dd13563f5c834551b7f1830266335364" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 36977 } } peers { permanent_uuid: "a7039556d1bb4f9991853edeb64b2a62" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 37679 } }
I20260807 18:13:21.041121 29842 catalog_manager.cc:5495] T 4ca8f5095320465d89d9dad5e011a234 P dd13563f5c834551b7f1830266335364 reported cstate change: term changed from 0 to 1, leader changed from <none> to dd13563f5c834551b7f1830266335364 (127.18.154.2). New cstate: current_term: 1 leader_uuid: "dd13563f5c834551b7f1830266335364" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f364d0e1a3f14b55829f9ee56ac06792" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 46809 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd13563f5c834551b7f1830266335364" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 36977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a7039556d1bb4f9991853edeb64b2a62" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 37679 } health_report { overall_health: UNKNOWN } } }
Aug 07 18:13:21 dist-test-slave-gkw7 krb5kdc[29758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126397, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.154.62@KRBTEST.COM for oryx/127.18.154.62@KRBTEST.COM
I20260807 18:13:21.152545 30689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:21.136167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:33361 (local address 127.0.0.1:50450)
0807 18:13:21.137050 (+ 883us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:21.137068 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 18:13:21.137226 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:13:21.137803 (+ 577us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:13:21.137807 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:13:21.138815 (+ 1008us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:13:21.139133 (+ 318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:21.139146 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:21.140581 (+ 1435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:21.140597 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:21.141800 (+ 1203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:21.141815 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:21.142680 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:21.142686 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:21.142933 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:21.143767 (+ 834us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:13:21.143793 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:13:21.146451 (+ 2658us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:13:21.150012 (+ 3561us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:13:21.150025 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:13:21.150032 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:13:21.150494 (+ 462us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:13:21.150920 (+ 426us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:13:21.150924 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:13:21.150926 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:13:21.151095 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:13:21.151650 (+ 555us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:13:21.151656 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:13:21.151839 (+ 183us) client_negotiation.cc:769] Sending connection context
0807 18:13:21.151985 (+ 146us) client_negotiation.cc:240] Negotiation successful
0807 18:13:21.152265 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":734,"thread_start_us":605,"threads_started":1}
I20260807 18:13:21.152895 30690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:21.136185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50450 (local address 127.18.154.62:33361)
0807 18:13:21.136961 (+ 776us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:21.136967 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:21.137166 (+ 199us) server_negotiation.cc:407] Connection header received
0807 18:13:21.137477 (+ 311us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:21.137482 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:21.137614 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:21.137770 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:21.139375 (+ 1605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:21.140413 (+ 1038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:21.141964 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:21.142463 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:21.142623 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:21.146717 (+ 4094us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:13:21.146726 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:13:21.146732 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:13:21.146784 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:13:21.149848 (+ 3064us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:21.150642 (+ 794us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:21.150647 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:21.150649 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:21.150796 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:21.151220 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:21.151224 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:21.151226 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:21.151517 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:13:21.151687 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0807 18:13:21.152080 (+ 393us) server_negotiation.cc:299] Negotiation successful
0807 18:13:21.152437 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":142,"server-negotiator.queue_time_us":587,"thread_start_us":350,"threads_started":1}
I20260807 18:13:21.158077 29842 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.18.154.62@KRBTEST.COM'} at 127.0.0.1:50450:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260807 18:13:21.159030 29842 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:13:21.166595 30057 tablet_service.cc:1467] Processing CreateTablet for tablet 4f63ca2156b1492982031ced07af3911 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=6c9ddb037aab428eb6701772af3a7fca]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 18:13:21.166738 30271 tablet_service.cc:1467] Processing CreateTablet for tablet 4f63ca2156b1492982031ced07af3911 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=6c9ddb037aab428eb6701772af3a7fca]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 18:13:21.167186 30057 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4f63ca2156b1492982031ced07af3911. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:13:21.167348 30271 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4f63ca2156b1492982031ced07af3911. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:13:21.172793 30482 tablet_service.cc:1467] Processing CreateTablet for tablet 4f63ca2156b1492982031ced07af3911 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=6c9ddb037aab428eb6701772af3a7fca]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 18:13:21.173159 30633 tablet_bootstrap.cc:492] T 4f63ca2156b1492982031ced07af3911 P f364d0e1a3f14b55829f9ee56ac06792: Bootstrap starting.
I20260807 18:13:21.173355 30482 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4f63ca2156b1492982031ced07af3911. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:13:21.174400 30632 tablet_bootstrap.cc:492] T 4f63ca2156b1492982031ced07af3911 P dd13563f5c834551b7f1830266335364: Bootstrap starting.
I20260807 18:13:21.174927 30633 tablet_bootstrap.cc:654] T 4f63ca2156b1492982031ced07af3911 P f364d0e1a3f14b55829f9ee56ac06792: Neither blocks nor log segments found. Creating new log.
I20260807 18:13:21.176234 30632 tablet_bootstrap.cc:654] T 4f63ca2156b1492982031ced07af3911 P dd13563f5c834551b7f1830266335364: Neither blocks nor log segments found. Creating new log.
I20260807 18:13:21.178834 30632 tablet_bootstrap.cc:492] T 4f63ca2156b1492982031ced07af3911 P dd13563f5c834551b7f1830266335364: No bootstrap required, opened a new log
I20260807 18:13:21.179004 30632 ts_tablet_manager.cc:1397] T 4f63ca2156b1492982031ced07af3911 P dd13563f5c834551b7f1830266335364: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20260807 18:13:21.179554 30631 tablet_bootstrap.cc:492] T 4f63ca2156b1492982031ced07af3911 P a7039556d1bb4f9991853edeb64b2a62: Bootstrap starting.
I20260807 18:13:21.179934 30633 tablet_bootstrap.cc:492] T 4f63ca2156b1492982031ced07af3911 P f364d0e1a3f14b55829f9ee56ac06792: No bootstrap required, opened a new log
I20260807 18:13:21.180186 30633 ts_tablet_manager.cc:1397] T 4f63ca2156b1492982031ced07af3911 P f364d0e1a3f14b55829f9ee56ac06792: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260807 18:13:21.180109 30632 raft_consensus.cc:348] T 4f63ca2156b1492982031ced07af3911 P dd13563f5c834551b7f1830266335364 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f364d0e1a3f14b55829f9ee56ac06792" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 46809 } } peers { permanent_uuid: "dd13563f5c834551b7f1830266335364" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 36977 } } peers { permanent_uuid: "a7039556d1bb4f9991853edeb64b2a62" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 37679 } }
I20260807 18:13:21.180480 30632 raft_consensus.cc:374] T 4f63ca2156b1492982031ced07af3911 P dd13563f5c834551b7f1830266335364 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:13:21.180569 30632 raft_consensus.cc:729] T 4f63ca2156b1492982031ced07af3911 P dd13563f5c834551b7f1830266335364 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd13563f5c834551b7f1830266335364, State: Initialized, Role: FOLLOWER
I20260807 18:13:21.180856 30632 consensus_queue.cc:260] T 4f63ca2156b1492982031ced07af3911 P dd13563f5c834551b7f1830266335364 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f364d0e1a3f14b55829f9ee56ac06792" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 46809 } } peers { permanent_uuid: "dd13563f5c834551b7f1830266335364" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 36977 } } peers { permanent_uuid: "a7039556d1bb4f9991853edeb64b2a62" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 37679 } }
I20260807 18:13:21.180996 30633 raft_consensus.cc:348] T 4f63ca2156b1492982031ced07af3911 P f364d0e1a3f14b55829f9ee56ac06792 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f364d0e1a3f14b55829f9ee56ac06792" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 46809 } } peers { permanent_uuid: "dd13563f5c834551b7f1830266335364" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 36977 } } peers { permanent_uuid: "a7039556d1bb4f9991853edeb64b2a62" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 37679 } }
I20260807 18:13:21.181270 30633 raft_consensus.cc:374] T 4f63ca2156b1492982031ced07af3911 P f364d0e1a3f14b55829f9ee56ac06792 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:13:21.181348 30631 tablet_bootstrap.cc:654] T 4f63ca2156b1492982031ced07af3911 P a7039556d1bb4f9991853edeb64b2a62: Neither blocks nor log segments found. Creating new log.
I20260807 18:13:21.181345 30633 raft_consensus.cc:729] T 4f63ca2156b1492982031ced07af3911 P f364d0e1a3f14b55829f9ee56ac06792 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f364d0e1a3f14b55829f9ee56ac06792, State: Initialized, Role: FOLLOWER
I20260807 18:13:21.181299 30641 tablet_replica.cc:439] T 4f63ca2156b1492982031ced07af3911 P dd13563f5c834551b7f1830266335364: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f364d0e1a3f14b55829f9ee56ac06792" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 46809 } } peers { permanent_uuid: "dd13563f5c834551b7f1830266335364" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 36977 } } peers { permanent_uuid: "a7039556d1bb4f9991853edeb64b2a62" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 37679 } } }
I20260807 18:13:21.181612 30641 tablet_replica.cc:442] T 4f63ca2156b1492982031ced07af3911 P dd13563f5c834551b7f1830266335364: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 18:13:21.181638 30633 consensus_queue.cc:260] T 4f63ca2156b1492982031ced07af3911 P f364d0e1a3f14b55829f9ee56ac06792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f364d0e1a3f14b55829f9ee56ac06792" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 46809 } } peers { permanent_uuid: "dd13563f5c834551b7f1830266335364" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 36977 } } peers { permanent_uuid: "a7039556d1bb4f9991853edeb64b2a62" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 37679 } }
I20260807 18:13:21.182096 30632 ts_tablet_manager.cc:1428] T 4f63ca2156b1492982031ced07af3911 P dd13563f5c834551b7f1830266335364: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260807 18:13:21.182093 30642 tablet_replica.cc:439] T 4f63ca2156b1492982031ced07af3911 P f364d0e1a3f14b55829f9ee56ac06792: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f364d0e1a3f14b55829f9ee56ac06792" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 46809 } } peers { permanent_uuid: "dd13563f5c834551b7f1830266335364" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 36977 } } peers { permanent_uuid: "a7039556d1bb4f9991853edeb64b2a62" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 37679 } } }
I20260807 18:13:21.182466 30642 tablet_replica.cc:442] T 4f63ca2156b1492982031ced07af3911 P f364d0e1a3f14b55829f9ee56ac06792: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 18:13:21.183948 30633 ts_tablet_manager.cc:1428] T 4f63ca2156b1492982031ced07af3911 P f364d0e1a3f14b55829f9ee56ac06792: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260807 18:13:21.187120 30631 tablet_bootstrap.cc:492] T 4f63ca2156b1492982031ced07af3911 P a7039556d1bb4f9991853edeb64b2a62: No bootstrap required, opened a new log
I20260807 18:13:21.187294 30631 ts_tablet_manager.cc:1397] T 4f63ca2156b1492982031ced07af3911 P a7039556d1bb4f9991853edeb64b2a62: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.003s
I20260807 18:13:21.187987 30631 raft_consensus.cc:348] T 4f63ca2156b1492982031ced07af3911 P a7039556d1bb4f9991853edeb64b2a62 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f364d0e1a3f14b55829f9ee56ac06792" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 46809 } } peers { permanent_uuid: "dd13563f5c834551b7f1830266335364" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 36977 } } peers { permanent_uuid: "a7039556d1bb4f9991853edeb64b2a62" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 37679 } }
I20260807 18:13:21.188234 30631 raft_consensus.cc:374] T 4f63ca2156b1492982031ced07af3911 P a7039556d1bb4f9991853edeb64b2a62 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:13:21.188337 30631 raft_consensus.cc:729] T 4f63ca2156b1492982031ced07af3911 P a7039556d1bb4f9991853edeb64b2a62 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a7039556d1bb4f9991853edeb64b2a62, State: Initialized, Role: FOLLOWER
I20260807 18:13:21.188586 30631 consensus_queue.cc:260] T 4f63ca2156b1492982031ced07af3911 P a7039556d1bb4f9991853edeb64b2a62 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f364d0e1a3f14b55829f9ee56ac06792" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 46809 } } peers { permanent_uuid: "dd13563f5c834551b7f1830266335364" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 36977 } } peers { permanent_uuid: "a7039556d1bb4f9991853edeb64b2a62" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 37679 } }
I20260807 18:13:21.189136 30631 ts_tablet_manager.cc:1428] T 4f63ca2156b1492982031ced07af3911 P a7039556d1bb4f9991853edeb64b2a62: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260807 18:13:21.189018 30640 tablet_replica.cc:439] T 4f63ca2156b1492982031ced07af3911 P a7039556d1bb4f9991853edeb64b2a62: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f364d0e1a3f14b55829f9ee56ac06792" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 46809 } } peers { permanent_uuid: "dd13563f5c834551b7f1830266335364" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 36977 } } peers { permanent_uuid: "a7039556d1bb4f9991853edeb64b2a62" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 37679 } } }
I20260807 18:13:21.189334 30640 tablet_replica.cc:442] T 4f63ca2156b1492982031ced07af3911 P a7039556d1bb4f9991853edeb64b2a62: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 18:13:21.476598 30640 raft_consensus.cc:482] T 4f63ca2156b1492982031ced07af3911 P a7039556d1bb4f9991853edeb64b2a62 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:13:21.476902 30640 raft_consensus.cc:504] T 4f63ca2156b1492982031ced07af3911 P a7039556d1bb4f9991853edeb64b2a62 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f364d0e1a3f14b55829f9ee56ac06792" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 46809 } } peers { permanent_uuid: "dd13563f5c834551b7f1830266335364" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 36977 } } peers { permanent_uuid: "a7039556d1bb4f9991853edeb64b2a62" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 37679 } }
I20260807 18:13:21.477967 30640 leader_election.cc:288] T 4f63ca2156b1492982031ced07af3911 P a7039556d1bb4f9991853edeb64b2a62 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f364d0e1a3f14b55829f9ee56ac06792 (127.18.154.1:46809), dd13563f5c834551b7f1830266335364 (127.18.154.2:36977)
I20260807 18:13:21.478703 30093 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f63ca2156b1492982031ced07af3911" candidate_uuid: "a7039556d1bb4f9991853edeb64b2a62" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f364d0e1a3f14b55829f9ee56ac06792" is_pre_election: true
I20260807 18:13:21.478729 30300 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f63ca2156b1492982031ced07af3911" candidate_uuid: "a7039556d1bb4f9991853edeb64b2a62" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd13563f5c834551b7f1830266335364" is_pre_election: true
I20260807 18:13:21.479036 30093 raft_consensus.cc:2454] T 4f63ca2156b1492982031ced07af3911 P f364d0e1a3f14b55829f9ee56ac06792 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a7039556d1bb4f9991853edeb64b2a62 in term 0.
I20260807 18:13:21.479015 30300 raft_consensus.cc:2454] T 4f63ca2156b1492982031ced07af3911 P dd13563f5c834551b7f1830266335364 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a7039556d1bb4f9991853edeb64b2a62 in term 0.
I20260807 18:13:21.479581 30406 leader_election.cc:302] T 4f63ca2156b1492982031ced07af3911 P a7039556d1bb4f9991853edeb64b2a62 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a7039556d1bb4f9991853edeb64b2a62, f364d0e1a3f14b55829f9ee56ac06792; no voters:
I20260807 18:13:21.479902 30640 raft_consensus.cc:2781] T 4f63ca2156b1492982031ced07af3911 P a7039556d1bb4f9991853edeb64b2a62 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:13:21.480044 30640 raft_consensus.cc:482] T 4f63ca2156b1492982031ced07af3911 P a7039556d1bb4f9991853edeb64b2a62 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:13:21.480144 30640 raft_consensus.cc:3037] T 4f63ca2156b1492982031ced07af3911 P a7039556d1bb4f9991853edeb64b2a62 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:13:21.482271 30640 raft_consensus.cc:504] T 4f63ca2156b1492982031ced07af3911 P a7039556d1bb4f9991853edeb64b2a62 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f364d0e1a3f14b55829f9ee56ac06792" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 46809 } } peers { permanent_uuid: "dd13563f5c834551b7f1830266335364" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 36977 } } peers { permanent_uuid: "a7039556d1bb4f9991853edeb64b2a62" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 37679 } }
I20260807 18:13:21.482846 30640 leader_election.cc:288] T 4f63ca2156b1492982031ced07af3911 P a7039556d1bb4f9991853edeb64b2a62 [CANDIDATE]: Term 1 election: Requested vote from peers f364d0e1a3f14b55829f9ee56ac06792 (127.18.154.1:46809), dd13563f5c834551b7f1830266335364 (127.18.154.2:36977)
I20260807 18:13:21.483438 30300 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f63ca2156b1492982031ced07af3911" candidate_uuid: "a7039556d1bb4f9991853edeb64b2a62" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd13563f5c834551b7f1830266335364"
I20260807 18:13:21.483418 30093 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f63ca2156b1492982031ced07af3911" candidate_uuid: "a7039556d1bb4f9991853edeb64b2a62" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f364d0e1a3f14b55829f9ee56ac06792"
I20260807 18:13:21.483678 30300 raft_consensus.cc:3037] T 4f63ca2156b1492982031ced07af3911 P dd13563f5c834551b7f1830266335364 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:13:21.483705 30093 raft_consensus.cc:3037] T 4f63ca2156b1492982031ced07af3911 P f364d0e1a3f14b55829f9ee56ac06792 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:13:21.486739 30093 raft_consensus.cc:2454] T 4f63ca2156b1492982031ced07af3911 P f364d0e1a3f14b55829f9ee56ac06792 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a7039556d1bb4f9991853edeb64b2a62 in term 1.
I20260807 18:13:21.486753 30300 raft_consensus.cc:2454] T 4f63ca2156b1492982031ced07af3911 P dd13563f5c834551b7f1830266335364 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a7039556d1bb4f9991853edeb64b2a62 in term 1.
I20260807 18:13:21.487254 30406 leader_election.cc:302] T 4f63ca2156b1492982031ced07af3911 P a7039556d1bb4f9991853edeb64b2a62 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: a7039556d1bb4f9991853edeb64b2a62, dd13563f5c834551b7f1830266335364, f364d0e1a3f14b55829f9ee56ac06792; no voters:
I20260807 18:13:21.487658 30640 raft_consensus.cc:2781] T 4f63ca2156b1492982031ced07af3911 P a7039556d1bb4f9991853edeb64b2a62 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:13:21.488067 30640 raft_consensus.cc:686] T 4f63ca2156b1492982031ced07af3911 P a7039556d1bb4f9991853edeb64b2a62 [term 1 LEADER]: Becoming Leader. State: Replica: a7039556d1bb4f9991853edeb64b2a62, State: Running, Role: LEADER
I20260807 18:13:21.488377 30640 consensus_queue.cc:237] T 4f63ca2156b1492982031ced07af3911 P a7039556d1bb4f9991853edeb64b2a62 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f364d0e1a3f14b55829f9ee56ac06792" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 46809 } } peers { permanent_uuid: "dd13563f5c834551b7f1830266335364" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 36977 } } peers { permanent_uuid: "a7039556d1bb4f9991853edeb64b2a62" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 37679 } }
I20260807 18:13:21.490105 30695 tablet_replica.cc:439] T 4f63ca2156b1492982031ced07af3911 P a7039556d1bb4f9991853edeb64b2a62: TxnStatusTablet state changed. Reason: New leader a7039556d1bb4f9991853edeb64b2a62. Latest consensus state: current_term: 1 leader_uuid: "a7039556d1bb4f9991853edeb64b2a62" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f364d0e1a3f14b55829f9ee56ac06792" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 46809 } } peers { permanent_uuid: "dd13563f5c834551b7f1830266335364" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 36977 } } peers { permanent_uuid: "a7039556d1bb4f9991853edeb64b2a62" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 37679 } } }
I20260807 18:13:21.490425 30695 tablet_replica.cc:442] T 4f63ca2156b1492982031ced07af3911 P a7039556d1bb4f9991853edeb64b2a62: This TxnStatusTablet replica's current role is: LEADER
I20260807 18:13:21.491365 30697 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260807 18:13:21.491612 30697 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260807 18:13:21.491894 29848 catalog_manager.cc:5495] T 4f63ca2156b1492982031ced07af3911 P a7039556d1bb4f9991853edeb64b2a62 reported cstate change: term changed from 0 to 1, leader changed from <none> to a7039556d1bb4f9991853edeb64b2a62 (127.18.154.3). New cstate: current_term: 1 leader_uuid: "a7039556d1bb4f9991853edeb64b2a62" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f364d0e1a3f14b55829f9ee56ac06792" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 46809 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd13563f5c834551b7f1830266335364" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 36977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a7039556d1bb4f9991853edeb64b2a62" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 37679 } health_report { overall_health: HEALTHY } } }
I20260807 18:13:21.548358 30641 consensus_queue.cc:1035] T 4ca8f5095320465d89d9dad5e011a234 P dd13563f5c834551b7f1830266335364 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a7039556d1bb4f9991853edeb64b2a62" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 37679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:13:21.550912 30641 consensus_queue.cc:1035] T 4ca8f5095320465d89d9dad5e011a234 P dd13563f5c834551b7f1830266335364 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f364d0e1a3f14b55829f9ee56ac06792" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 46809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:13:21.554105 30689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:21.545510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.3:37679 (local address 127.0.0.1:45220)
0807 18:13:21.545994 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:21.546020 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0807 18:13:21.546198 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:13:21.546667 (+ 469us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:13:21.546673 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:13:21.546688 (+ 15us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 18:13:21.546898 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:21.546910 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:21.548605 (+ 1695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:21.548612 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:21.550314 (+ 1702us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:21.550334 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:21.551929 (+ 1595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:21.551940 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:21.552278 (+ 338us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:21.552399 (+ 121us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 18:13:21.553177 (+ 778us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 18:13:21.553213 (+ 36us) client_negotiation.cc:769] Sending connection context
0807 18:13:21.553295 (+ 82us) client_negotiation.cc:240] Negotiation successful
0807 18:13:21.553585 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253}
I20260807 18:13:21.555580 30704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:21.545584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45220 (local address 127.18.154.3:37679)
0807 18:13:21.546269 (+ 685us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:21.546275 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:21.546301 (+ 26us) server_negotiation.cc:407] Connection header received
0807 18:13:21.546389 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:21.546395 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:21.546496 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:21.546624 (+ 128us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:13:21.547095 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:21.548406 (+ 1311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:21.550537 (+ 2131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:21.551654 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:21.552678 (+ 1024us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:21.552835 (+ 157us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:13:21.553020 (+ 185us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:13:21.554841 (+ 1821us) server_negotiation.cc:1035] Waiting for connection context
0807 18:13:21.554951 (+ 110us) server_negotiation.cc:299] Negotiation successful
0807 18:13:21.555077 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":532,"thread_start_us":780,"threads_started":1}
I20260807 18:13:21.560698 30300 raft_consensus.cc:1261] T 4f63ca2156b1492982031ced07af3911 P dd13563f5c834551b7f1830266335364 [term 1 FOLLOWER]: Refusing update from remote peer a7039556d1bb4f9991853edeb64b2a62: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:13:21.561017 30092 raft_consensus.cc:1261] T 4f63ca2156b1492982031ced07af3911 P f364d0e1a3f14b55829f9ee56ac06792 [term 1 FOLLOWER]: Refusing update from remote peer a7039556d1bb4f9991853edeb64b2a62: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:13:21.561615 30640 consensus_queue.cc:1035] T 4f63ca2156b1492982031ced07af3911 P a7039556d1bb4f9991853edeb64b2a62 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd13563f5c834551b7f1830266335364" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 36977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:13:21.562302 30640 consensus_queue.cc:1035] T 4f63ca2156b1492982031ced07af3911 P a7039556d1bb4f9991853edeb64b2a62 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f364d0e1a3f14b55829f9ee56ac06792" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 46809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:13:21.578485 30641 tablet_replica.cc:439] T 4f63ca2156b1492982031ced07af3911 P dd13563f5c834551b7f1830266335364: TxnStatusTablet state changed. Reason: New leader a7039556d1bb4f9991853edeb64b2a62. Latest consensus state: current_term: 1 leader_uuid: "a7039556d1bb4f9991853edeb64b2a62" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f364d0e1a3f14b55829f9ee56ac06792" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 46809 } } peers { permanent_uuid: "dd13563f5c834551b7f1830266335364" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 36977 } } peers { permanent_uuid: "a7039556d1bb4f9991853edeb64b2a62" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 37679 } } }
I20260807 18:13:21.578812 30641 tablet_replica.cc:442] T 4f63ca2156b1492982031ced07af3911 P dd13563f5c834551b7f1830266335364: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 18:13:21.591254 30720 mvcc.cc:205] Tried to move back new op lower bound from 7315973740786520064 to 7315973740498862080. Current Snapshot: MvccSnapshot[applied={T|T < 7315973740786520064}]
I20260807 18:13:21.598196 30641 tablet_replica.cc:439] T 4f63ca2156b1492982031ced07af3911 P dd13563f5c834551b7f1830266335364: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a7039556d1bb4f9991853edeb64b2a62" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f364d0e1a3f14b55829f9ee56ac06792" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 46809 } } peers { permanent_uuid: "dd13563f5c834551b7f1830266335364" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 36977 } } peers { permanent_uuid: "a7039556d1bb4f9991853edeb64b2a62" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 37679 } } }
I20260807 18:13:21.598584 30641 tablet_replica.cc:442] T 4f63ca2156b1492982031ced07af3911 P dd13563f5c834551b7f1830266335364: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 18:13:21.601361 30718 tablet_replica.cc:439] T 4f63ca2156b1492982031ced07af3911 P a7039556d1bb4f9991853edeb64b2a62: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a7039556d1bb4f9991853edeb64b2a62" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f364d0e1a3f14b55829f9ee56ac06792" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 46809 } } peers { permanent_uuid: "dd13563f5c834551b7f1830266335364" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 36977 } } peers { permanent_uuid: "a7039556d1bb4f9991853edeb64b2a62" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 37679 } } }
I20260807 18:13:21.601889 30718 tablet_replica.cc:442] T 4f63ca2156b1492982031ced07af3911 P a7039556d1bb4f9991853edeb64b2a62: This TxnStatusTablet replica's current role is: LEADER
I20260807 18:13:21.614950 30642 tablet_replica.cc:439] T 4f63ca2156b1492982031ced07af3911 P f364d0e1a3f14b55829f9ee56ac06792: TxnStatusTablet state changed. Reason: New leader a7039556d1bb4f9991853edeb64b2a62. Latest consensus state: current_term: 1 leader_uuid: "a7039556d1bb4f9991853edeb64b2a62" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f364d0e1a3f14b55829f9ee56ac06792" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 46809 } } peers { permanent_uuid: "dd13563f5c834551b7f1830266335364" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 36977 } } peers { permanent_uuid: "a7039556d1bb4f9991853edeb64b2a62" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 37679 } } }
I20260807 18:13:21.615374 30642 tablet_replica.cc:442] T 4f63ca2156b1492982031ced07af3911 P f364d0e1a3f14b55829f9ee56ac06792: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 18:13:21.617372 30640 tablet_replica.cc:439] T 4f63ca2156b1492982031ced07af3911 P a7039556d1bb4f9991853edeb64b2a62: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a7039556d1bb4f9991853edeb64b2a62" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f364d0e1a3f14b55829f9ee56ac06792" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 46809 } } peers { permanent_uuid: "dd13563f5c834551b7f1830266335364" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 36977 } } peers { permanent_uuid: "a7039556d1bb4f9991853edeb64b2a62" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 37679 } } }
I20260807 18:13:21.617743 30640 tablet_replica.cc:442] T 4f63ca2156b1492982031ced07af3911 P a7039556d1bb4f9991853edeb64b2a62: This TxnStatusTablet replica's current role is: LEADER
I20260807 18:13:21.626786 30642 tablet_replica.cc:439] T 4f63ca2156b1492982031ced07af3911 P f364d0e1a3f14b55829f9ee56ac06792: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a7039556d1bb4f9991853edeb64b2a62" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f364d0e1a3f14b55829f9ee56ac06792" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 46809 } } peers { permanent_uuid: "dd13563f5c834551b7f1830266335364" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 36977 } } peers { permanent_uuid: "a7039556d1bb4f9991853edeb64b2a62" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 37679 } } }
I20260807 18:13:21.627177 30642 tablet_replica.cc:442] T 4f63ca2156b1492982031ced07af3911 P f364d0e1a3f14b55829f9ee56ac06792: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 18:13:21.645339 30749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:21.616850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58564 (local address 127.18.154.2:36977)
0807 18:13:21.637402 (+ 20552us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:21.637409 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:21.637437 (+ 28us) server_negotiation.cc:407] Connection header received
0807 18:13:21.637588 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:21.637595 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:21.637798 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:21.638043 (+ 245us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:13:21.638607 (+ 564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:21.640047 (+ 1440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:21.642129 (+ 2082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:21.642934 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:21.643098 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:21.643647 (+ 549us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:13:21.644523 (+ 876us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:13:21.644693 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0807 18:13:21.644807 (+ 114us) server_negotiation.cc:299] Negotiation successful
0807 18:13:21.644907 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20360,"thread_start_us":20155,"threads_started":1}
I20260807 18:13:21.669633 30704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:21.658430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.2:34015 (local address 127.18.154.3:37679)
0807 18:13:21.658657 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:21.658665 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:21.662350 (+ 3685us) server_negotiation.cc:407] Connection header received
0807 18:13:21.662560 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:21.662575 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:21.662673 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:21.662792 (+ 119us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:13:21.663503 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:21.664781 (+ 1278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:21.667705 (+ 2924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:21.668157 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:21.668263 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:21.668846 (+ 583us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:13:21.668957 (+ 111us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:13:21.669031 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0807 18:13:21.669262 (+ 231us) server_negotiation.cc:299] Negotiation successful
0807 18:13:21.669337 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260807 18:13:21.669812 30758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:21.658391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.3:37679 (local address 127.18.154.2:34015)
0807 18:13:21.662212 (+ 3821us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:21.662234 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 18:13:21.662403 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:13:21.662831 (+ 428us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:13:21.662842 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:13:21.662857 (+ 15us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 18:13:21.663276 (+ 419us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:21.663289 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:21.665443 (+ 2154us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:21.665450 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:21.667514 (+ 2064us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:21.667533 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:21.668338 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:21.668345 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:21.668549 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:21.668641 (+ 92us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 18:13:21.669072 (+ 431us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 18:13:21.669086 (+ 14us) client_negotiation.cc:769] Sending connection context
0807 18:13:21.669182 (+ 96us) client_negotiation.cc:240] Negotiation successful
0807 18:13:21.669350 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3606,"thread_start_us":3376,"threads_started":1}
I20260807 18:13:21.714303 30749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:21.707492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.3:58253 (local address 127.18.154.2:36977)
0807 18:13:21.707706 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:21.707712 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:21.707924 (+ 212us) server_negotiation.cc:407] Connection header received
0807 18:13:21.708189 (+ 265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:21.708195 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:21.708282 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:21.708384 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:13:21.708962 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:21.710034 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:21.712001 (+ 1967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:21.712748 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:21.712877 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:21.713511 (+ 634us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:13:21.713696 (+ 185us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:13:21.713804 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0807 18:13:21.713929 (+ 125us) server_negotiation.cc:299] Negotiation successful
0807 18:13:21.714008 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260807 18:13:21.714555 30760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:21.706902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.2:36977 (local address 127.18.154.3:58253)
0807 18:13:21.707816 (+ 914us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:21.707833 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 18:13:21.708007 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:13:21.708403 (+ 396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:13:21.708408 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:13:21.708420 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 18:13:21.708787 (+ 367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:21.708801 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:21.710177 (+ 1376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:21.710182 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:21.711822 (+ 1640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:21.711839 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:21.713018 (+ 1179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:21.713025 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:21.713222 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:21.713307 (+ 85us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 18:13:21.713813 (+ 506us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 18:13:21.713828 (+ 15us) client_negotiation.cc:769] Sending connection context
0807 18:13:21.713908 (+ 80us) client_negotiation.cc:240] Negotiation successful
0807 18:13:21.714060 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":688,"thread_start_us":473,"threads_started":1}
I20260807 18:13:21.767261 29848 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50446:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:13:21.767570 29848 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50446:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:13:21.771339 29848 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e61bfa7995fe4afcbcaf0939bac87929: Sending DeleteTablet for 3 replicas of tablet 4ca8f5095320465d89d9dad5e011a234
I20260807 18:13:21.772238 30057 tablet_service.cc:1514] Processing DeleteTablet for tablet 4ca8f5095320465d89d9dad5e011a234 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:13:21 UTC) from {username='oryx', principal='oryx/127.18.154.62@KRBTEST.COM'} at 127.0.0.1:60300
I20260807 18:13:21.772689 30271 tablet_service.cc:1514] Processing DeleteTablet for tablet 4ca8f5095320465d89d9dad5e011a234 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:13:21 UTC) from {username='oryx', principal='oryx/127.18.154.62@KRBTEST.COM'} at 127.0.0.1:58554
I20260807 18:13:21.772869 30482 tablet_service.cc:1514] Processing DeleteTablet for tablet 4ca8f5095320465d89d9dad5e011a234 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:13:21 UTC) from {username='oryx', principal='oryx/127.18.154.62@KRBTEST.COM'} at 127.0.0.1:45198
I20260807 18:13:21.785092 30765 tablet_replica.cc:330] T 4ca8f5095320465d89d9dad5e011a234 P dd13563f5c834551b7f1830266335364: stopping tablet replica
I20260807 18:13:21.785492 30765 raft_consensus.cc:2229] T 4ca8f5095320465d89d9dad5e011a234 P dd13563f5c834551b7f1830266335364 [term 1 LEADER]: Raft consensus shutting down.
I20260807 18:13:21.786403 30765 raft_consensus.cc:2258] T 4ca8f5095320465d89d9dad5e011a234 P dd13563f5c834551b7f1830266335364 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:13:21.788053 30765 ts_tablet_manager.cc:1905] T 4ca8f5095320465d89d9dad5e011a234 P dd13563f5c834551b7f1830266335364: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:13:21.788625 30763 tablet_replica.cc:330] T 4ca8f5095320465d89d9dad5e011a234 P f364d0e1a3f14b55829f9ee56ac06792: stopping tablet replica
I20260807 18:13:21.788909 30763 raft_consensus.cc:2229] T 4ca8f5095320465d89d9dad5e011a234 P f364d0e1a3f14b55829f9ee56ac06792 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:13:21.794590 30763 raft_consensus.cc:2258] T 4ca8f5095320465d89d9dad5e011a234 P f364d0e1a3f14b55829f9ee56ac06792 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:13:21.796023 30763 ts_tablet_manager.cc:1905] T 4ca8f5095320465d89d9dad5e011a234 P f364d0e1a3f14b55829f9ee56ac06792: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:13:21.799340 30766 tablet_replica.cc:330] T 4ca8f5095320465d89d9dad5e011a234 P a7039556d1bb4f9991853edeb64b2a62: stopping tablet replica
I20260807 18:13:21.799727 30766 raft_consensus.cc:2229] T 4ca8f5095320465d89d9dad5e011a234 P a7039556d1bb4f9991853edeb64b2a62 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:13:21.800113 30766 raft_consensus.cc:2258] T 4ca8f5095320465d89d9dad5e011a234 P a7039556d1bb4f9991853edeb64b2a62 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:13:21.801396 30766 ts_tablet_manager.cc:1905] T 4ca8f5095320465d89d9dad5e011a234 P a7039556d1bb4f9991853edeb64b2a62: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:13:21.809477 30766 ts_tablet_manager.cc:1918] T 4ca8f5095320465d89d9dad5e011a234 P a7039556d1bb4f9991853edeb64b2a62: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 18:13:21.809775 30766 log.cc:1196] T 4ca8f5095320465d89d9dad5e011a234 P a7039556d1bb4f9991853edeb64b2a62: Deleting WAL directory at /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126328808105-19048-0/minicluster-data/ts-2/wal/wals/4ca8f5095320465d89d9dad5e011a234
I20260807 18:13:21.810422 30766 ts_tablet_manager.cc:1939] T 4ca8f5095320465d89d9dad5e011a234 P a7039556d1bb4f9991853edeb64b2a62: Deleting consensus metadata
I20260807 18:13:21.811923 29823 catalog_manager.cc:4841] TS a7039556d1bb4f9991853edeb64b2a62 (127.18.154.3:37679): tablet 4ca8f5095320465d89d9dad5e011a234 (table test-table [id=4da3f1990d6f4bb79796a2d5fbe62e37]) successfully deleted
I20260807 18:13:21.814666 30765 ts_tablet_manager.cc:1918] T 4ca8f5095320465d89d9dad5e011a234 P dd13563f5c834551b7f1830266335364: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 18:13:21.814924 30765 log.cc:1196] T 4ca8f5095320465d89d9dad5e011a234 P dd13563f5c834551b7f1830266335364: Deleting WAL directory at /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126328808105-19048-0/minicluster-data/ts-1/wal/wals/4ca8f5095320465d89d9dad5e011a234
I20260807 18:13:21.815517 30765 ts_tablet_manager.cc:1939] T 4ca8f5095320465d89d9dad5e011a234 P dd13563f5c834551b7f1830266335364: Deleting consensus metadata
I20260807 18:13:21.816834 30763 ts_tablet_manager.cc:1918] T 4ca8f5095320465d89d9dad5e011a234 P f364d0e1a3f14b55829f9ee56ac06792: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 18:13:21.816999 30763 log.cc:1196] T 4ca8f5095320465d89d9dad5e011a234 P f364d0e1a3f14b55829f9ee56ac06792: Deleting WAL directory at /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126328808105-19048-0/minicluster-data/ts-0/wal/wals/4ca8f5095320465d89d9dad5e011a234
I20260807 18:13:21.816974 29819 catalog_manager.cc:4841] TS dd13563f5c834551b7f1830266335364 (127.18.154.2:36977): tablet 4ca8f5095320465d89d9dad5e011a234 (table test-table [id=4da3f1990d6f4bb79796a2d5fbe62e37]) successfully deleted
I20260807 18:13:21.817474 30763 ts_tablet_manager.cc:1939] T 4ca8f5095320465d89d9dad5e011a234 P f364d0e1a3f14b55829f9ee56ac06792: Deleting consensus metadata
I20260807 18:13:21.818882 29821 catalog_manager.cc:4841] TS f364d0e1a3f14b55829f9ee56ac06792 (127.18.154.1:46809): tablet 4ca8f5095320465d89d9dad5e011a234 (table test-table [id=4da3f1990d6f4bb79796a2d5fbe62e37]) successfully deleted
Aug 07 18:13:21 dist-test-slave-gkw7 krb5kdc[29758](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126401, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 18:13:21 dist-test-slave-gkw7 krb5kdc[29758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.18.154.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:13:21 dist-test-slave-gkw7 krb5kdc[29758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.18.154.62@KRBTEST.COM, Server not found in Kerberos database
W20260807 18:13:21.897936 30777 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:13:21.867485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50452 (local address 127.18.154.62:33361)
0807 18:13:21.883543 (+ 16058us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:21.883552 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:21.883578 (+ 26us) server_negotiation.cc:407] Connection header received
0807 18:13:21.883652 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:21.883657 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:21.883755 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:21.883899 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:21.885909 (+ 2010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:21.887135 (+ 1226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:21.889205 (+ 2070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:21.889956 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:21.890123 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:21.897598 (+ 7475us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50452: BlockingRecv error: recv got EOF from 127.0.0.1:50452 (error 108)
Metrics: {"server-negotiator.queue_time_us":15883,"thread_start_us":15881,"threads_started":1}
Aug 07 18:13:21 dist-test-slave-gkw7 krb5kdc[29758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126401, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.18.154.62@KRBTEST.COM
I20260807 18:13:21.925463 30777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:21.903830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50458 (local address 127.18.154.62:33361)
0807 18:13:21.904015 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:21.904023 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:21.906189 (+ 2166us) server_negotiation.cc:407] Connection header received
0807 18:13:21.906386 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:21.906395 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:21.906489 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:21.906615 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:21.908265 (+ 1650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:21.909444 (+ 1179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:21.911444 (+ 2000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:21.912269 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:21.912400 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:21.917920 (+ 5520us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:13:21.917927 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:13:21.917933 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:13:21.917987 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:13:21.921940 (+ 3953us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:21.922870 (+ 930us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:21.922877 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:21.922881 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:21.923080 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:21.923547 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:21.923553 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:21.923557 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:21.923943 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:13:21.924235 (+ 292us) server_negotiation.cc:1035] Waiting for connection context
0807 18:13:21.924584 (+ 349us) server_negotiation.cc:299] Negotiation successful
0807 18:13:21.924840 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260807 18:13:21.929997 29848 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50458:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 18:13:21.930855 29848 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:13:21.939725 30057 tablet_service.cc:1467] Processing CreateTablet for tablet 2fb774d54a7d4df5b40d35cc3427c107 (DEFAULT_TABLE table=test-table [id=7270ea9bd36e4ae6bf66501445af0956]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:13:21.940236 30057 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2fb774d54a7d4df5b40d35cc3427c107. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:13:21.942553 30482 tablet_service.cc:1467] Processing CreateTablet for tablet 2fb774d54a7d4df5b40d35cc3427c107 (DEFAULT_TABLE table=test-table [id=7270ea9bd36e4ae6bf66501445af0956]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:13:21.943049 30482 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2fb774d54a7d4df5b40d35cc3427c107. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:13:21.950666 30271 tablet_service.cc:1467] Processing CreateTablet for tablet 2fb774d54a7d4df5b40d35cc3427c107 (DEFAULT_TABLE table=test-table [id=7270ea9bd36e4ae6bf66501445af0956]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:13:21.951226 30271 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2fb774d54a7d4df5b40d35cc3427c107. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:13:21.955130 30784 tablet_bootstrap.cc:492] T 2fb774d54a7d4df5b40d35cc3427c107 P f364d0e1a3f14b55829f9ee56ac06792: Bootstrap starting.
I20260807 18:13:21.957988 30784 tablet_bootstrap.cc:654] T 2fb774d54a7d4df5b40d35cc3427c107 P f364d0e1a3f14b55829f9ee56ac06792: Neither blocks nor log segments found. Creating new log.
I20260807 18:13:21.971400 30784 tablet_bootstrap.cc:492] T 2fb774d54a7d4df5b40d35cc3427c107 P f364d0e1a3f14b55829f9ee56ac06792: No bootstrap required, opened a new log
I20260807 18:13:21.971732 30784 ts_tablet_manager.cc:1397] T 2fb774d54a7d4df5b40d35cc3427c107 P f364d0e1a3f14b55829f9ee56ac06792: Time spent bootstrapping tablet: real 0.017s user 0.011s sys 0.004s
I20260807 18:13:21.972695 30784 raft_consensus.cc:348] T 2fb774d54a7d4df5b40d35cc3427c107 P f364d0e1a3f14b55829f9ee56ac06792 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f364d0e1a3f14b55829f9ee56ac06792" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 46809 } } peers { permanent_uuid: "dd13563f5c834551b7f1830266335364" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 36977 } } peers { permanent_uuid: "a7039556d1bb4f9991853edeb64b2a62" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 37679 } }
I20260807 18:13:21.973212 30784 raft_consensus.cc:374] T 2fb774d54a7d4df5b40d35cc3427c107 P f364d0e1a3f14b55829f9ee56ac06792 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:13:21.973366 30784 raft_consensus.cc:729] T 2fb774d54a7d4df5b40d35cc3427c107 P f364d0e1a3f14b55829f9ee56ac06792 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f364d0e1a3f14b55829f9ee56ac06792, State: Initialized, Role: FOLLOWER
I20260807 18:13:21.973722 30784 consensus_queue.cc:260] T 2fb774d54a7d4df5b40d35cc3427c107 P f364d0e1a3f14b55829f9ee56ac06792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f364d0e1a3f14b55829f9ee56ac06792" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 46809 } } peers { permanent_uuid: "dd13563f5c834551b7f1830266335364" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 36977 } } peers { permanent_uuid: "a7039556d1bb4f9991853edeb64b2a62" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 37679 } }
I20260807 18:13:21.977223 30784 ts_tablet_manager.cc:1428] T 2fb774d54a7d4df5b40d35cc3427c107 P f364d0e1a3f14b55829f9ee56ac06792: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260807 18:13:21.978616 30790 tablet_bootstrap.cc:492] T 2fb774d54a7d4df5b40d35cc3427c107 P dd13563f5c834551b7f1830266335364: Bootstrap starting.
I20260807 18:13:21.981554 30790 tablet_bootstrap.cc:654] T 2fb774d54a7d4df5b40d35cc3427c107 P dd13563f5c834551b7f1830266335364: Neither blocks nor log segments found. Creating new log.
I20260807 18:13:21.982187 30786 tablet_bootstrap.cc:492] T 2fb774d54a7d4df5b40d35cc3427c107 P a7039556d1bb4f9991853edeb64b2a62: Bootstrap starting.
I20260807 18:13:21.984503 30786 tablet_bootstrap.cc:654] T 2fb774d54a7d4df5b40d35cc3427c107 P a7039556d1bb4f9991853edeb64b2a62: Neither blocks nor log segments found. Creating new log.
I20260807 18:13:21.991257 30790 tablet_bootstrap.cc:492] T 2fb774d54a7d4df5b40d35cc3427c107 P dd13563f5c834551b7f1830266335364: No bootstrap required, opened a new log
I20260807 18:13:21.991536 30790 ts_tablet_manager.cc:1397] T 2fb774d54a7d4df5b40d35cc3427c107 P dd13563f5c834551b7f1830266335364: Time spent bootstrapping tablet: real 0.013s user 0.008s sys 0.003s
I20260807 18:13:21.992550 30790 raft_consensus.cc:348] T 2fb774d54a7d4df5b40d35cc3427c107 P dd13563f5c834551b7f1830266335364 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f364d0e1a3f14b55829f9ee56ac06792" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 46809 } } peers { permanent_uuid: "dd13563f5c834551b7f1830266335364" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 36977 } } peers { permanent_uuid: "a7039556d1bb4f9991853edeb64b2a62" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 37679 } }
I20260807 18:13:21.992905 30790 raft_consensus.cc:374] T 2fb774d54a7d4df5b40d35cc3427c107 P dd13563f5c834551b7f1830266335364 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:13:21.993067 30790 raft_consensus.cc:729] T 2fb774d54a7d4df5b40d35cc3427c107 P dd13563f5c834551b7f1830266335364 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd13563f5c834551b7f1830266335364, State: Initialized, Role: FOLLOWER
I20260807 18:13:21.993459 30790 consensus_queue.cc:260] T 2fb774d54a7d4df5b40d35cc3427c107 P dd13563f5c834551b7f1830266335364 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f364d0e1a3f14b55829f9ee56ac06792" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 46809 } } peers { permanent_uuid: "dd13563f5c834551b7f1830266335364" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 36977 } } peers { permanent_uuid: "a7039556d1bb4f9991853edeb64b2a62" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 37679 } }
I20260807 18:13:21.997735 30790 ts_tablet_manager.cc:1428] T 2fb774d54a7d4df5b40d35cc3427c107 P dd13563f5c834551b7f1830266335364: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20260807 18:13:22.002328 30786 tablet_bootstrap.cc:492] T 2fb774d54a7d4df5b40d35cc3427c107 P a7039556d1bb4f9991853edeb64b2a62: No bootstrap required, opened a new log
I20260807 18:13:22.002596 30786 ts_tablet_manager.cc:1397] T 2fb774d54a7d4df5b40d35cc3427c107 P a7039556d1bb4f9991853edeb64b2a62: Time spent bootstrapping tablet: real 0.021s user 0.007s sys 0.009s
I20260807 18:13:22.003603 30786 raft_consensus.cc:348] T 2fb774d54a7d4df5b40d35cc3427c107 P a7039556d1bb4f9991853edeb64b2a62 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f364d0e1a3f14b55829f9ee56ac06792" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 46809 } } peers { permanent_uuid: "dd13563f5c834551b7f1830266335364" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 36977 } } peers { permanent_uuid: "a7039556d1bb4f9991853edeb64b2a62" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 37679 } }
I20260807 18:13:22.003922 30786 raft_consensus.cc:374] T 2fb774d54a7d4df5b40d35cc3427c107 P a7039556d1bb4f9991853edeb64b2a62 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:13:22.004081 30786 raft_consensus.cc:729] T 2fb774d54a7d4df5b40d35cc3427c107 P a7039556d1bb4f9991853edeb64b2a62 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a7039556d1bb4f9991853edeb64b2a62, State: Initialized, Role: FOLLOWER
I20260807 18:13:22.004400 30786 consensus_queue.cc:260] T 2fb774d54a7d4df5b40d35cc3427c107 P a7039556d1bb4f9991853edeb64b2a62 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f364d0e1a3f14b55829f9ee56ac06792" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 46809 } } peers { permanent_uuid: "dd13563f5c834551b7f1830266335364" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 36977 } } peers { permanent_uuid: "a7039556d1bb4f9991853edeb64b2a62" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 37679 } }
I20260807 18:13:22.005103 30786 ts_tablet_manager.cc:1428] T 2fb774d54a7d4df5b40d35cc3427c107 P a7039556d1bb4f9991853edeb64b2a62: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260807 18:13:22.174926 30719 raft_consensus.cc:482] T 2fb774d54a7d4df5b40d35cc3427c107 P a7039556d1bb4f9991853edeb64b2a62 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:13:22.175210 30719 raft_consensus.cc:504] T 2fb774d54a7d4df5b40d35cc3427c107 P a7039556d1bb4f9991853edeb64b2a62 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f364d0e1a3f14b55829f9ee56ac06792" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 46809 } } peers { permanent_uuid: "dd13563f5c834551b7f1830266335364" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 36977 } } peers { permanent_uuid: "a7039556d1bb4f9991853edeb64b2a62" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 37679 } }
I20260807 18:13:22.175961 30719 leader_election.cc:288] T 2fb774d54a7d4df5b40d35cc3427c107 P a7039556d1bb4f9991853edeb64b2a62 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f364d0e1a3f14b55829f9ee56ac06792 (127.18.154.1:46809), dd13563f5c834551b7f1830266335364 (127.18.154.2:36977)
I20260807 18:13:22.176698 30093 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2fb774d54a7d4df5b40d35cc3427c107" candidate_uuid: "a7039556d1bb4f9991853edeb64b2a62" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f364d0e1a3f14b55829f9ee56ac06792" is_pre_election: true
I20260807 18:13:22.176676 30300 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2fb774d54a7d4df5b40d35cc3427c107" candidate_uuid: "a7039556d1bb4f9991853edeb64b2a62" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd13563f5c834551b7f1830266335364" is_pre_election: true
I20260807 18:13:22.176960 30093 raft_consensus.cc:2454] T 2fb774d54a7d4df5b40d35cc3427c107 P f364d0e1a3f14b55829f9ee56ac06792 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a7039556d1bb4f9991853edeb64b2a62 in term 0.
I20260807 18:13:22.176970 30300 raft_consensus.cc:2454] T 2fb774d54a7d4df5b40d35cc3427c107 P dd13563f5c834551b7f1830266335364 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a7039556d1bb4f9991853edeb64b2a62 in term 0.
I20260807 18:13:22.177601 30406 leader_election.cc:302] T 2fb774d54a7d4df5b40d35cc3427c107 P a7039556d1bb4f9991853edeb64b2a62 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a7039556d1bb4f9991853edeb64b2a62, f364d0e1a3f14b55829f9ee56ac06792; no voters:
I20260807 18:13:22.177940 30719 raft_consensus.cc:2781] T 2fb774d54a7d4df5b40d35cc3427c107 P a7039556d1bb4f9991853edeb64b2a62 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:13:22.178081 30719 raft_consensus.cc:482] T 2fb774d54a7d4df5b40d35cc3427c107 P a7039556d1bb4f9991853edeb64b2a62 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:13:22.178190 30719 raft_consensus.cc:3037] T 2fb774d54a7d4df5b40d35cc3427c107 P a7039556d1bb4f9991853edeb64b2a62 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:13:22.181423 30719 raft_consensus.cc:504] T 2fb774d54a7d4df5b40d35cc3427c107 P a7039556d1bb4f9991853edeb64b2a62 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f364d0e1a3f14b55829f9ee56ac06792" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 46809 } } peers { permanent_uuid: "dd13563f5c834551b7f1830266335364" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 36977 } } peers { permanent_uuid: "a7039556d1bb4f9991853edeb64b2a62" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 37679 } }
I20260807 18:13:22.182194 30719 leader_election.cc:288] T 2fb774d54a7d4df5b40d35cc3427c107 P a7039556d1bb4f9991853edeb64b2a62 [CANDIDATE]: Term 1 election: Requested vote from peers f364d0e1a3f14b55829f9ee56ac06792 (127.18.154.1:46809), dd13563f5c834551b7f1830266335364 (127.18.154.2:36977)
I20260807 18:13:22.182763 30300 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2fb774d54a7d4df5b40d35cc3427c107" candidate_uuid: "a7039556d1bb4f9991853edeb64b2a62" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd13563f5c834551b7f1830266335364"
I20260807 18:13:22.182796 30093 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2fb774d54a7d4df5b40d35cc3427c107" candidate_uuid: "a7039556d1bb4f9991853edeb64b2a62" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f364d0e1a3f14b55829f9ee56ac06792"
I20260807 18:13:22.183000 30300 raft_consensus.cc:3037] T 2fb774d54a7d4df5b40d35cc3427c107 P dd13563f5c834551b7f1830266335364 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:13:22.183036 30093 raft_consensus.cc:3037] T 2fb774d54a7d4df5b40d35cc3427c107 P f364d0e1a3f14b55829f9ee56ac06792 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:13:22.185324 30300 raft_consensus.cc:2454] T 2fb774d54a7d4df5b40d35cc3427c107 P dd13563f5c834551b7f1830266335364 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a7039556d1bb4f9991853edeb64b2a62 in term 1.
I20260807 18:13:22.185832 30404 leader_election.cc:302] T 2fb774d54a7d4df5b40d35cc3427c107 P a7039556d1bb4f9991853edeb64b2a62 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a7039556d1bb4f9991853edeb64b2a62, dd13563f5c834551b7f1830266335364; no voters:
I20260807 18:13:22.186069 30093 raft_consensus.cc:2454] T 2fb774d54a7d4df5b40d35cc3427c107 P f364d0e1a3f14b55829f9ee56ac06792 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a7039556d1bb4f9991853edeb64b2a62 in term 1.
I20260807 18:13:22.186123 30719 raft_consensus.cc:2781] T 2fb774d54a7d4df5b40d35cc3427c107 P a7039556d1bb4f9991853edeb64b2a62 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:13:22.186300 30719 raft_consensus.cc:686] T 2fb774d54a7d4df5b40d35cc3427c107 P a7039556d1bb4f9991853edeb64b2a62 [term 1 LEADER]: Becoming Leader. State: Replica: a7039556d1bb4f9991853edeb64b2a62, State: Running, Role: LEADER
I20260807 18:13:22.186697 30719 consensus_queue.cc:237] T 2fb774d54a7d4df5b40d35cc3427c107 P a7039556d1bb4f9991853edeb64b2a62 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f364d0e1a3f14b55829f9ee56ac06792" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 46809 } } peers { permanent_uuid: "dd13563f5c834551b7f1830266335364" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 36977 } } peers { permanent_uuid: "a7039556d1bb4f9991853edeb64b2a62" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 37679 } }
I20260807 18:13:22.189873 29848 catalog_manager.cc:5495] T 2fb774d54a7d4df5b40d35cc3427c107 P a7039556d1bb4f9991853edeb64b2a62 reported cstate change: term changed from 0 to 1, leader changed from <none> to a7039556d1bb4f9991853edeb64b2a62 (127.18.154.3). New cstate: current_term: 1 leader_uuid: "a7039556d1bb4f9991853edeb64b2a62" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f364d0e1a3f14b55829f9ee56ac06792" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 46809 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd13563f5c834551b7f1830266335364" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 36977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a7039556d1bb4f9991853edeb64b2a62" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 37679 } health_report { overall_health: HEALTHY } } }
I20260807 18:13:22.216482 30797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:22.208600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45224 (local address 127.18.154.3:37679)
0807 18:13:22.210472 (+ 1872us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:22.210477 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:22.210498 (+ 21us) server_negotiation.cc:407] Connection header received
0807 18:13:22.210571 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:22.210575 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:22.210660 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:22.210800 (+ 140us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:13:22.211306 (+ 506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:22.212241 (+ 935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:22.214078 (+ 1837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:22.214658 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:22.214820 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:22.215486 (+ 666us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:13:22.215635 (+ 149us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:13:22.215762 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0807 18:13:22.215983 (+ 221us) server_negotiation.cc:299] Negotiation successful
0807 18:13:22.216080 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1754,"thread_start_us":1637,"threads_started":1}
I20260807 18:13:22.219923 30300 raft_consensus.cc:1261] T 2fb774d54a7d4df5b40d35cc3427c107 P dd13563f5c834551b7f1830266335364 [term 1 FOLLOWER]: Refusing update from remote peer a7039556d1bb4f9991853edeb64b2a62: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:13:22.219990 30093 raft_consensus.cc:1261] T 2fb774d54a7d4df5b40d35cc3427c107 P f364d0e1a3f14b55829f9ee56ac06792 [term 1 FOLLOWER]: Refusing update from remote peer a7039556d1bb4f9991853edeb64b2a62: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:13:22.220794 30695 consensus_queue.cc:1035] T 2fb774d54a7d4df5b40d35cc3427c107 P a7039556d1bb4f9991853edeb64b2a62 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd13563f5c834551b7f1830266335364" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 36977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:13:22.221514 30719 consensus_queue.cc:1035] T 2fb774d54a7d4df5b40d35cc3427c107 P a7039556d1bb4f9991853edeb64b2a62 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f364d0e1a3f14b55829f9ee56ac06792" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 46809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:13:22.235690 29848 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50458:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:13:22.236042 29848 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50458:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:13:22.240480 29848 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e61bfa7995fe4afcbcaf0939bac87929: Sending DeleteTablet for 3 replicas of tablet 2fb774d54a7d4df5b40d35cc3427c107
I20260807 18:13:22.241671 30057 tablet_service.cc:1514] Processing DeleteTablet for tablet 2fb774d54a7d4df5b40d35cc3427c107 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:13:22 UTC) from {username='oryx', principal='oryx/127.18.154.62@KRBTEST.COM'} at 127.0.0.1:60300
I20260807 18:13:22.242049 30763 tablet_replica.cc:330] T 2fb774d54a7d4df5b40d35cc3427c107 P f364d0e1a3f14b55829f9ee56ac06792: stopping tablet replica
I20260807 18:13:22.242261 30763 raft_consensus.cc:2229] T 2fb774d54a7d4df5b40d35cc3427c107 P f364d0e1a3f14b55829f9ee56ac06792 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:13:22.242272 30482 tablet_service.cc:1514] Processing DeleteTablet for tablet 2fb774d54a7d4df5b40d35cc3427c107 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:13:22 UTC) from {username='oryx', principal='oryx/127.18.154.62@KRBTEST.COM'} at 127.0.0.1:45198
I20260807 18:13:22.242411 30763 pending_rounds.cc:62] T 2fb774d54a7d4df5b40d35cc3427c107 P f364d0e1a3f14b55829f9ee56ac06792: Trying to abort 2 pending ops.
I20260807 18:13:22.242674 30766 tablet_replica.cc:330] T 2fb774d54a7d4df5b40d35cc3427c107 P a7039556d1bb4f9991853edeb64b2a62: stopping tablet replica
I20260807 18:13:22.242924 30766 raft_consensus.cc:2229] T 2fb774d54a7d4df5b40d35cc3427c107 P a7039556d1bb4f9991853edeb64b2a62 [term 1 LEADER]: Raft consensus shutting down.
I20260807 18:13:22.243381 30766 raft_consensus.cc:2258] T 2fb774d54a7d4df5b40d35cc3427c107 P a7039556d1bb4f9991853edeb64b2a62 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:13:22.243083 30763 pending_rounds.cc:68] T 2fb774d54a7d4df5b40d35cc3427c107 P f364d0e1a3f14b55829f9ee56ac06792: Aborting op as it isn't in flight: id { term: 1 index: 2 } timestamp: 7315973743484788736 op_type: WRITE_OP write_request { tablet_id: "2fb774d54a7d4df5b40d35cc3427c107" schema { columns { name: "key" type: INT32 is_key: true is_nullable: false immutable: false } columns { name: "val" type: INT32 is_key: false is_nullable: true immutable: false } } row_operations { rows: "<redacted>""\001\003\00090\000\0001\324\000\000" indirect_data: "<redacted>""" } external_consistency_mode: CLIENT_PROPAGATED authz_token { token_data: "\010\356\302\330\323\006\"8\n\ntest-admin\022*\n 7270ea9bd36e4ae6bf66501445af0956\020\001\030\001 \001(\001" signature: "<redacted>""Y\023\022\214\305)\366b^y\332\357\266\336\315T\372S\367\335TB\274\330\276X\260\364HG\366) \247+\"\367\257\374+\242\314\203\204\320\350\274D\205w\260\303\016I\357\331\025\365\217\023_at\223" signing_key_seq_num: 0 } } request_id { client_id: "126dfc06dcb647d2857c1b038e80176c" seq_no: 0 first_incomplete_seq_no: 0 attempt_no: 0 }
I20260807 18:13:22.244207 30763 pending_rounds.cc:68] T 2fb774d54a7d4df5b40d35cc3427c107 P f364d0e1a3f14b55829f9ee56ac06792: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7315973743359528960 op_type: NO_OP noop_request { }
I20260807 18:13:22.244427 30763 raft_consensus.cc:2866] T 2fb774d54a7d4df5b40d35cc3427c107 P f364d0e1a3f14b55829f9ee56ac06792 [term 1 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260807 18:13:22.244606 30763 raft_consensus.cc:2258] T 2fb774d54a7d4df5b40d35cc3427c107 P f364d0e1a3f14b55829f9ee56ac06792 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:13:22.246407 30766 ts_tablet_manager.cc:1905] T 2fb774d54a7d4df5b40d35cc3427c107 P a7039556d1bb4f9991853edeb64b2a62: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:13:22.247025 30763 ts_tablet_manager.cc:1905] T 2fb774d54a7d4df5b40d35cc3427c107 P f364d0e1a3f14b55829f9ee56ac06792: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:13:22.249788 30271 tablet_service.cc:1514] Processing DeleteTablet for tablet 2fb774d54a7d4df5b40d35cc3427c107 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:13:22 UTC) from {username='oryx', principal='oryx/127.18.154.62@KRBTEST.COM'} at 127.0.0.1:58554
I20260807 18:13:22.250170 30765 tablet_replica.cc:330] T 2fb774d54a7d4df5b40d35cc3427c107 P dd13563f5c834551b7f1830266335364: stopping tablet replica
I20260807 18:13:22.250424 30765 raft_consensus.cc:2229] T 2fb774d54a7d4df5b40d35cc3427c107 P dd13563f5c834551b7f1830266335364 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:13:22.250752 30765 raft_consensus.cc:2258] T 2fb774d54a7d4df5b40d35cc3427c107 P dd13563f5c834551b7f1830266335364 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:13:22.254802 30763 ts_tablet_manager.cc:1918] T 2fb774d54a7d4df5b40d35cc3427c107 P f364d0e1a3f14b55829f9ee56ac06792: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:13:22.255015 30763 log.cc:1196] T 2fb774d54a7d4df5b40d35cc3427c107 P f364d0e1a3f14b55829f9ee56ac06792: Deleting WAL directory at /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126328808105-19048-0/minicluster-data/ts-0/wal/wals/2fb774d54a7d4df5b40d35cc3427c107
I20260807 18:13:22.255569 30763 ts_tablet_manager.cc:1939] T 2fb774d54a7d4df5b40d35cc3427c107 P f364d0e1a3f14b55829f9ee56ac06792: Deleting consensus metadata
I20260807 18:13:22.255533 29848 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50458:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 18:13:22.256955 29821 catalog_manager.cc:4841] TS f364d0e1a3f14b55829f9ee56ac06792 (127.18.154.1:46809): tablet 2fb774d54a7d4df5b40d35cc3427c107 (table test-table [id=7270ea9bd36e4ae6bf66501445af0956]) successfully deleted
W20260807 18:13:22.257794 29848 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:13:22.262462 30765 ts_tablet_manager.cc:1905] T 2fb774d54a7d4df5b40d35cc3427c107 P dd13563f5c834551b7f1830266335364: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:13:22.263332 30766 ts_tablet_manager.cc:1918] T 2fb774d54a7d4df5b40d35cc3427c107 P a7039556d1bb4f9991853edeb64b2a62: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:13:22.263518 30766 log.cc:1196] T 2fb774d54a7d4df5b40d35cc3427c107 P a7039556d1bb4f9991853edeb64b2a62: Deleting WAL directory at /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126328808105-19048-0/minicluster-data/ts-2/wal/wals/2fb774d54a7d4df5b40d35cc3427c107
I20260807 18:13:22.264087 30766 ts_tablet_manager.cc:1939] T 2fb774d54a7d4df5b40d35cc3427c107 P a7039556d1bb4f9991853edeb64b2a62: Deleting consensus metadata
I20260807 18:13:22.265897 29823 catalog_manager.cc:4841] TS a7039556d1bb4f9991853edeb64b2a62 (127.18.154.3:37679): tablet 2fb774d54a7d4df5b40d35cc3427c107 (table test-table [id=7270ea9bd36e4ae6bf66501445af0956]) successfully deleted
I20260807 18:13:22.269668 30057 tablet_service.cc:1467] Processing CreateTablet for tablet 385081bc5bff49af8acc777d47c5978f (DEFAULT_TABLE table=test-table [id=58a1faa89f99440db475570690d1dbb5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:13:22.269814 30271 tablet_service.cc:1467] Processing CreateTablet for tablet 385081bc5bff49af8acc777d47c5978f (DEFAULT_TABLE table=test-table [id=58a1faa89f99440db475570690d1dbb5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:13:22.270263 30057 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 385081bc5bff49af8acc777d47c5978f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:13:22.270326 30271 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 385081bc5bff49af8acc777d47c5978f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:13:22.271740 30765 ts_tablet_manager.cc:1918] T 2fb774d54a7d4df5b40d35cc3427c107 P dd13563f5c834551b7f1830266335364: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:13:22.271968 30765 log.cc:1196] T 2fb774d54a7d4df5b40d35cc3427c107 P dd13563f5c834551b7f1830266335364: Deleting WAL directory at /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126328808105-19048-0/minicluster-data/ts-1/wal/wals/2fb774d54a7d4df5b40d35cc3427c107
I20260807 18:13:22.272701 30765 ts_tablet_manager.cc:1939] T 2fb774d54a7d4df5b40d35cc3427c107 P dd13563f5c834551b7f1830266335364: Deleting consensus metadata
I20260807 18:13:22.274008 29819 catalog_manager.cc:4841] TS dd13563f5c834551b7f1830266335364 (127.18.154.2:36977): tablet 2fb774d54a7d4df5b40d35cc3427c107 (table test-table [id=7270ea9bd36e4ae6bf66501445af0956]) successfully deleted
I20260807 18:13:22.274482 30482 tablet_service.cc:1467] Processing CreateTablet for tablet 385081bc5bff49af8acc777d47c5978f (DEFAULT_TABLE table=test-table [id=58a1faa89f99440db475570690d1dbb5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:13:22.274930 30482 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 385081bc5bff49af8acc777d47c5978f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:13:22.278882 30790 tablet_bootstrap.cc:492] T 385081bc5bff49af8acc777d47c5978f P dd13563f5c834551b7f1830266335364: Bootstrap starting.
I20260807 18:13:22.281215 30790 tablet_bootstrap.cc:654] T 385081bc5bff49af8acc777d47c5978f P dd13563f5c834551b7f1830266335364: Neither blocks nor log segments found. Creating new log.
I20260807 18:13:22.281586 30784 tablet_bootstrap.cc:492] T 385081bc5bff49af8acc777d47c5978f P f364d0e1a3f14b55829f9ee56ac06792: Bootstrap starting.
I20260807 18:13:22.283278 30784 tablet_bootstrap.cc:654] T 385081bc5bff49af8acc777d47c5978f P f364d0e1a3f14b55829f9ee56ac06792: Neither blocks nor log segments found. Creating new log.
I20260807 18:13:22.284325 30790 tablet_bootstrap.cc:492] T 385081bc5bff49af8acc777d47c5978f P dd13563f5c834551b7f1830266335364: No bootstrap required, opened a new log
I20260807 18:13:22.284632 30790 ts_tablet_manager.cc:1397] T 385081bc5bff49af8acc777d47c5978f P dd13563f5c834551b7f1830266335364: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.002s
I20260807 18:13:22.285621 30790 raft_consensus.cc:348] T 385081bc5bff49af8acc777d47c5978f P dd13563f5c834551b7f1830266335364 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f364d0e1a3f14b55829f9ee56ac06792" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 46809 } } peers { permanent_uuid: "dd13563f5c834551b7f1830266335364" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 36977 } } peers { permanent_uuid: "a7039556d1bb4f9991853edeb64b2a62" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 37679 } }
I20260807 18:13:22.285867 30786 tablet_bootstrap.cc:492] T 385081bc5bff49af8acc777d47c5978f P a7039556d1bb4f9991853edeb64b2a62: Bootstrap starting.
I20260807 18:13:22.286077 30790 raft_consensus.cc:374] T 385081bc5bff49af8acc777d47c5978f P dd13563f5c834551b7f1830266335364 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:13:22.286190 30790 raft_consensus.cc:729] T 385081bc5bff49af8acc777d47c5978f P dd13563f5c834551b7f1830266335364 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd13563f5c834551b7f1830266335364, State: Initialized, Role: FOLLOWER
I20260807 18:13:22.286526 30790 consensus_queue.cc:260] T 385081bc5bff49af8acc777d47c5978f P dd13563f5c834551b7f1830266335364 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f364d0e1a3f14b55829f9ee56ac06792" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 46809 } } peers { permanent_uuid: "dd13563f5c834551b7f1830266335364" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 36977 } } peers { permanent_uuid: "a7039556d1bb4f9991853edeb64b2a62" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 37679 } }
I20260807 18:13:22.288040 30786 tablet_bootstrap.cc:654] T 385081bc5bff49af8acc777d47c5978f P a7039556d1bb4f9991853edeb64b2a62: Neither blocks nor log segments found. Creating new log.
I20260807 18:13:22.290443 30790 ts_tablet_manager.cc:1428] T 385081bc5bff49af8acc777d47c5978f P dd13563f5c834551b7f1830266335364: Time spent starting tablet: real 0.006s user 0.000s sys 0.002s
I20260807 18:13:22.294027 30786 tablet_bootstrap.cc:492] T 385081bc5bff49af8acc777d47c5978f P a7039556d1bb4f9991853edeb64b2a62: No bootstrap required, opened a new log
I20260807 18:13:22.294252 30786 ts_tablet_manager.cc:1397] T 385081bc5bff49af8acc777d47c5978f P a7039556d1bb4f9991853edeb64b2a62: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.004s
I20260807 18:13:22.295382 30786 raft_consensus.cc:348] T 385081bc5bff49af8acc777d47c5978f P a7039556d1bb4f9991853edeb64b2a62 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f364d0e1a3f14b55829f9ee56ac06792" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 46809 } } peers { permanent_uuid: "dd13563f5c834551b7f1830266335364" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 36977 } } peers { permanent_uuid: "a7039556d1bb4f9991853edeb64b2a62" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 37679 } }
I20260807 18:13:22.295800 30786 raft_consensus.cc:374] T 385081bc5bff49af8acc777d47c5978f P a7039556d1bb4f9991853edeb64b2a62 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:13:22.295849 30784 tablet_bootstrap.cc:492] T 385081bc5bff49af8acc777d47c5978f P f364d0e1a3f14b55829f9ee56ac06792: No bootstrap required, opened a new log
I20260807 18:13:22.295914 30786 raft_consensus.cc:729] T 385081bc5bff49af8acc777d47c5978f P a7039556d1bb4f9991853edeb64b2a62 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a7039556d1bb4f9991853edeb64b2a62, State: Initialized, Role: FOLLOWER
I20260807 18:13:22.296020 30784 ts_tablet_manager.cc:1397] T 385081bc5bff49af8acc777d47c5978f P f364d0e1a3f14b55829f9ee56ac06792: Time spent bootstrapping tablet: real 0.015s user 0.008s sys 0.003s
I20260807 18:13:22.296219 30786 consensus_queue.cc:260] T 385081bc5bff49af8acc777d47c5978f P a7039556d1bb4f9991853edeb64b2a62 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f364d0e1a3f14b55829f9ee56ac06792" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 46809 } } peers { permanent_uuid: "dd13563f5c834551b7f1830266335364" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 36977 } } peers { permanent_uuid: "a7039556d1bb4f9991853edeb64b2a62" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 37679 } }
I20260807 18:13:22.296867 30784 raft_consensus.cc:348] T 385081bc5bff49af8acc777d47c5978f P f364d0e1a3f14b55829f9ee56ac06792 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f364d0e1a3f14b55829f9ee56ac06792" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 46809 } } peers { permanent_uuid: "dd13563f5c834551b7f1830266335364" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 36977 } } peers { permanent_uuid: "a7039556d1bb4f9991853edeb64b2a62" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 37679 } }
I20260807 18:13:22.297029 30786 ts_tablet_manager.cc:1428] T 385081bc5bff49af8acc777d47c5978f P a7039556d1bb4f9991853edeb64b2a62: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260807 18:13:22.297216 30784 raft_consensus.cc:374] T 385081bc5bff49af8acc777d47c5978f P f364d0e1a3f14b55829f9ee56ac06792 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:13:22.297305 30784 raft_consensus.cc:729] T 385081bc5bff49af8acc777d47c5978f P f364d0e1a3f14b55829f9ee56ac06792 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f364d0e1a3f14b55829f9ee56ac06792, State: Initialized, Role: FOLLOWER
I20260807 18:13:22.297583 30784 consensus_queue.cc:260] T 385081bc5bff49af8acc777d47c5978f P f364d0e1a3f14b55829f9ee56ac06792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f364d0e1a3f14b55829f9ee56ac06792" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 46809 } } peers { permanent_uuid: "dd13563f5c834551b7f1830266335364" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 36977 } } peers { permanent_uuid: "a7039556d1bb4f9991853edeb64b2a62" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 37679 } }
I20260807 18:13:22.298878 30784 ts_tablet_manager.cc:1428] T 385081bc5bff49af8acc777d47c5978f P f364d0e1a3f14b55829f9ee56ac06792: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260807 18:13:22.320083 30719 raft_consensus.cc:482] T 385081bc5bff49af8acc777d47c5978f P a7039556d1bb4f9991853edeb64b2a62 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:13:22.320318 30719 raft_consensus.cc:504] T 385081bc5bff49af8acc777d47c5978f P a7039556d1bb4f9991853edeb64b2a62 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f364d0e1a3f14b55829f9ee56ac06792" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 46809 } } peers { permanent_uuid: "dd13563f5c834551b7f1830266335364" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 36977 } } peers { permanent_uuid: "a7039556d1bb4f9991853edeb64b2a62" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 37679 } }
I20260807 18:13:22.321033 30719 leader_election.cc:288] T 385081bc5bff49af8acc777d47c5978f P a7039556d1bb4f9991853edeb64b2a62 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f364d0e1a3f14b55829f9ee56ac06792 (127.18.154.1:46809), dd13563f5c834551b7f1830266335364 (127.18.154.2:36977)
I20260807 18:13:22.321722 30300 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "385081bc5bff49af8acc777d47c5978f" candidate_uuid: "a7039556d1bb4f9991853edeb64b2a62" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd13563f5c834551b7f1830266335364" is_pre_election: true
I20260807 18:13:22.321697 30092 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "385081bc5bff49af8acc777d47c5978f" candidate_uuid: "a7039556d1bb4f9991853edeb64b2a62" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f364d0e1a3f14b55829f9ee56ac06792" is_pre_election: true
I20260807 18:13:22.322017 30300 raft_consensus.cc:2454] T 385081bc5bff49af8acc777d47c5978f P dd13563f5c834551b7f1830266335364 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a7039556d1bb4f9991853edeb64b2a62 in term 0.
I20260807 18:13:22.322119 30092 raft_consensus.cc:2454] T 385081bc5bff49af8acc777d47c5978f P f364d0e1a3f14b55829f9ee56ac06792 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a7039556d1bb4f9991853edeb64b2a62 in term 0.
I20260807 18:13:22.322482 30404 leader_election.cc:302] T 385081bc5bff49af8acc777d47c5978f P a7039556d1bb4f9991853edeb64b2a62 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a7039556d1bb4f9991853edeb64b2a62, dd13563f5c834551b7f1830266335364; no voters:
I20260807 18:13:22.322850 30719 raft_consensus.cc:2781] T 385081bc5bff49af8acc777d47c5978f P a7039556d1bb4f9991853edeb64b2a62 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:13:22.323063 30719 raft_consensus.cc:482] T 385081bc5bff49af8acc777d47c5978f P a7039556d1bb4f9991853edeb64b2a62 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:13:22.323179 30719 raft_consensus.cc:3037] T 385081bc5bff49af8acc777d47c5978f P a7039556d1bb4f9991853edeb64b2a62 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:13:22.325261 30719 raft_consensus.cc:504] T 385081bc5bff49af8acc777d47c5978f P a7039556d1bb4f9991853edeb64b2a62 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f364d0e1a3f14b55829f9ee56ac06792" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 46809 } } peers { permanent_uuid: "dd13563f5c834551b7f1830266335364" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 36977 } } peers { permanent_uuid: "a7039556d1bb4f9991853edeb64b2a62" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 37679 } }
I20260807 18:13:22.325994 30719 leader_election.cc:288] T 385081bc5bff49af8acc777d47c5978f P a7039556d1bb4f9991853edeb64b2a62 [CANDIDATE]: Term 1 election: Requested vote from peers f364d0e1a3f14b55829f9ee56ac06792 (127.18.154.1:46809), dd13563f5c834551b7f1830266335364 (127.18.154.2:36977)
I20260807 18:13:22.326459 30092 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "385081bc5bff49af8acc777d47c5978f" candidate_uuid: "a7039556d1bb4f9991853edeb64b2a62" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f364d0e1a3f14b55829f9ee56ac06792"
I20260807 18:13:22.326512 30300 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "385081bc5bff49af8acc777d47c5978f" candidate_uuid: "a7039556d1bb4f9991853edeb64b2a62" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd13563f5c834551b7f1830266335364"
I20260807 18:13:22.326683 30092 raft_consensus.cc:3037] T 385081bc5bff49af8acc777d47c5978f P f364d0e1a3f14b55829f9ee56ac06792 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:13:22.326794 30300 raft_consensus.cc:3037] T 385081bc5bff49af8acc777d47c5978f P dd13563f5c834551b7f1830266335364 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:13:22.328925 30092 raft_consensus.cc:2454] T 385081bc5bff49af8acc777d47c5978f P f364d0e1a3f14b55829f9ee56ac06792 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a7039556d1bb4f9991853edeb64b2a62 in term 1.
I20260807 18:13:22.329305 30300 raft_consensus.cc:2454] T 385081bc5bff49af8acc777d47c5978f P dd13563f5c834551b7f1830266335364 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a7039556d1bb4f9991853edeb64b2a62 in term 1.
I20260807 18:13:22.329357 30406 leader_election.cc:302] T 385081bc5bff49af8acc777d47c5978f P a7039556d1bb4f9991853edeb64b2a62 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a7039556d1bb4f9991853edeb64b2a62, f364d0e1a3f14b55829f9ee56ac06792; no voters:
I20260807 18:13:22.329706 30719 raft_consensus.cc:2781] T 385081bc5bff49af8acc777d47c5978f P a7039556d1bb4f9991853edeb64b2a62 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:13:22.329888 30719 raft_consensus.cc:686] T 385081bc5bff49af8acc777d47c5978f P a7039556d1bb4f9991853edeb64b2a62 [term 1 LEADER]: Becoming Leader. State: Replica: a7039556d1bb4f9991853edeb64b2a62, State: Running, Role: LEADER
I20260807 18:13:22.330202 30719 consensus_queue.cc:237] T 385081bc5bff49af8acc777d47c5978f P a7039556d1bb4f9991853edeb64b2a62 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f364d0e1a3f14b55829f9ee56ac06792" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 46809 } } peers { permanent_uuid: "dd13563f5c834551b7f1830266335364" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 36977 } } peers { permanent_uuid: "a7039556d1bb4f9991853edeb64b2a62" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 37679 } }
I20260807 18:13:22.332624 29842 catalog_manager.cc:5495] T 385081bc5bff49af8acc777d47c5978f P a7039556d1bb4f9991853edeb64b2a62 reported cstate change: term changed from 0 to 1, leader changed from <none> to a7039556d1bb4f9991853edeb64b2a62 (127.18.154.3). New cstate: current_term: 1 leader_uuid: "a7039556d1bb4f9991853edeb64b2a62" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f364d0e1a3f14b55829f9ee56ac06792" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 46809 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd13563f5c834551b7f1830266335364" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 36977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a7039556d1bb4f9991853edeb64b2a62" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 37679 } health_report { overall_health: HEALTHY } } }
I20260807 18:13:22.363909 30797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:22.358856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.3:52921 (local address 127.18.154.3:37679)
0807 18:13:22.359028 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:22.359034 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:22.359598 (+ 564us) server_negotiation.cc:407] Connection header received
0807 18:13:22.359741 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:22.359744 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:22.359804 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:22.359879 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:13:22.360311 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:22.361003 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:22.362394 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:22.362787 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:22.362879 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:22.363235 (+ 356us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:13:22.363343 (+ 108us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:13:22.363410 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0807 18:13:22.363553 (+ 143us) server_negotiation.cc:299] Negotiation successful
0807 18:13:22.363615 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260807 18:13:22.363943 30812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:22.358835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.3:37679 (local address 127.18.154.3:52921)
0807 18:13:22.359503 (+ 668us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:22.359519 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 18:13:22.359638 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:13:22.359907 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:13:22.359909 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:13:22.359918 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 18:13:22.360173 (+ 255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:22.360180 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:22.361114 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:22.361118 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:22.362257 (+ 1139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:22.362269 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:22.362933 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:22.362937 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:22.363058 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:22.363091 (+ 33us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 18:13:22.363436 (+ 345us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 18:13:22.363446 (+ 10us) client_negotiation.cc:769] Sending connection context
0807 18:13:22.363511 (+ 65us) client_negotiation.cc:240] Negotiation successful
0807 18:13:22.363612 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":535,"thread_start_us":411,"threads_started":1}
I20260807 18:13:22.372512 30300 raft_consensus.cc:1261] T 385081bc5bff49af8acc777d47c5978f P dd13563f5c834551b7f1830266335364 [term 1 FOLLOWER]: Refusing update from remote peer a7039556d1bb4f9991853edeb64b2a62: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:13:22.372525 30092 raft_consensus.cc:1261] T 385081bc5bff49af8acc777d47c5978f P f364d0e1a3f14b55829f9ee56ac06792 [term 1 FOLLOWER]: Refusing update from remote peer a7039556d1bb4f9991853edeb64b2a62: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:13:22.373054 30718 consensus_queue.cc:1035] T 385081bc5bff49af8acc777d47c5978f P a7039556d1bb4f9991853edeb64b2a62 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd13563f5c834551b7f1830266335364" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 36977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:13:22.373351 30719 consensus_queue.cc:1035] T 385081bc5bff49af8acc777d47c5978f P a7039556d1bb4f9991853edeb64b2a62 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f364d0e1a3f14b55829f9ee56ac06792" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 46809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:13:22.434412 29842 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50458:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:13:22.434655 29842 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50458:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:13:22.437686 29842 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e61bfa7995fe4afcbcaf0939bac87929: Sending DeleteTablet for 3 replicas of tablet 385081bc5bff49af8acc777d47c5978f
I20260807 18:13:22.438536 30057 tablet_service.cc:1514] Processing DeleteTablet for tablet 385081bc5bff49af8acc777d47c5978f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:13:22 UTC) from {username='oryx', principal='oryx/127.18.154.62@KRBTEST.COM'} at 127.0.0.1:60300
I20260807 18:13:22.438629 30271 tablet_service.cc:1514] Processing DeleteTablet for tablet 385081bc5bff49af8acc777d47c5978f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:13:22 UTC) from {username='oryx', principal='oryx/127.18.154.62@KRBTEST.COM'} at 127.0.0.1:58554
I20260807 18:13:22.438714 30480 tablet_service.cc:1514] Processing DeleteTablet for tablet 385081bc5bff49af8acc777d47c5978f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:13:22 UTC) from {username='oryx', principal='oryx/127.18.154.62@KRBTEST.COM'} at 127.0.0.1:45198
I20260807 18:13:22.438889 30763 tablet_replica.cc:330] T 385081bc5bff49af8acc777d47c5978f P f364d0e1a3f14b55829f9ee56ac06792: stopping tablet replica
I20260807 18:13:22.439111 30765 tablet_replica.cc:330] T 385081bc5bff49af8acc777d47c5978f P dd13563f5c834551b7f1830266335364: stopping tablet replica
I20260807 18:13:22.439131 30763 raft_consensus.cc:2229] T 385081bc5bff49af8acc777d47c5978f P f364d0e1a3f14b55829f9ee56ac06792 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:13:22.439332 30765 raft_consensus.cc:2229] T 385081bc5bff49af8acc777d47c5978f P dd13563f5c834551b7f1830266335364 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:13:22.439361 30763 raft_consensus.cc:2258] T 385081bc5bff49af8acc777d47c5978f P f364d0e1a3f14b55829f9ee56ac06792 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:13:22.439585 30765 raft_consensus.cc:2258] T 385081bc5bff49af8acc777d47c5978f P dd13563f5c834551b7f1830266335364 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:13:22.439781 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 29953
I20260807 18:13:22.439896 30766 tablet_replica.cc:330] T 385081bc5bff49af8acc777d47c5978f P a7039556d1bb4f9991853edeb64b2a62: stopping tablet replica
I20260807 18:13:22.440080 30766 raft_consensus.cc:2229] T 385081bc5bff49af8acc777d47c5978f P a7039556d1bb4f9991853edeb64b2a62 [term 1 LEADER]: Raft consensus shutting down.
I20260807 18:13:22.440441 30766 raft_consensus.cc:2258] T 385081bc5bff49af8acc777d47c5978f P a7039556d1bb4f9991853edeb64b2a62 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:13:22.440601 30763 ts_tablet_manager.cc:1905] T 385081bc5bff49af8acc777d47c5978f P f364d0e1a3f14b55829f9ee56ac06792: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:13:22.442200 30765 ts_tablet_manager.cc:1905] T 385081bc5bff49af8acc777d47c5978f P dd13563f5c834551b7f1830266335364: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:13:22.442819 30766 ts_tablet_manager.cc:1905] T 385081bc5bff49af8acc777d47c5978f P a7039556d1bb4f9991853edeb64b2a62: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:13:22.448943 30765 ts_tablet_manager.cc:1918] T 385081bc5bff49af8acc777d47c5978f P dd13563f5c834551b7f1830266335364: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 18:13:22.449093 30765 log.cc:1196] T 385081bc5bff49af8acc777d47c5978f P dd13563f5c834551b7f1830266335364: Deleting WAL directory at /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126328808105-19048-0/minicluster-data/ts-1/wal/wals/385081bc5bff49af8acc777d47c5978f
I20260807 18:13:22.449429 30766 ts_tablet_manager.cc:1918] T 385081bc5bff49af8acc777d47c5978f P a7039556d1bb4f9991853edeb64b2a62: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 18:13:22.449543 30765 ts_tablet_manager.cc:1939] T 385081bc5bff49af8acc777d47c5978f P dd13563f5c834551b7f1830266335364: Deleting consensus metadata
I20260807 18:13:22.449589 30766 log.cc:1196] T 385081bc5bff49af8acc777d47c5978f P a7039556d1bb4f9991853edeb64b2a62: Deleting WAL directory at /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126328808105-19048-0/minicluster-data/ts-2/wal/wals/385081bc5bff49af8acc777d47c5978f
I20260807 18:13:22.449999 30766 ts_tablet_manager.cc:1939] T 385081bc5bff49af8acc777d47c5978f P a7039556d1bb4f9991853edeb64b2a62: Deleting consensus metadata
I20260807 18:13:22.450723 29819 catalog_manager.cc:4841] TS dd13563f5c834551b7f1830266335364 (127.18.154.2:36977): tablet 385081bc5bff49af8acc777d47c5978f (table test-table [id=58a1faa89f99440db475570690d1dbb5]) successfully deleted
I20260807 18:13:22.451037 29823 catalog_manager.cc:4841] TS a7039556d1bb4f9991853edeb64b2a62 (127.18.154.3:37679): tablet 385081bc5bff49af8acc777d47c5978f (table test-table [id=58a1faa89f99440db475570690d1dbb5]) successfully deleted
I20260807 18:13:22.453629 30763 ts_tablet_manager.cc:1918] T 385081bc5bff49af8acc777d47c5978f P f364d0e1a3f14b55829f9ee56ac06792: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 18:13:22.453822 30763 log.cc:1196] T 385081bc5bff49af8acc777d47c5978f P f364d0e1a3f14b55829f9ee56ac06792: Deleting WAL directory at /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786126328808105-19048-0/minicluster-data/ts-0/wal/wals/385081bc5bff49af8acc777d47c5978f
I20260807 18:13:22.454268 30763 ts_tablet_manager.cc:1939] T 385081bc5bff49af8acc777d47c5978f P f364d0e1a3f14b55829f9ee56ac06792: Deleting consensus metadata
I20260807 18:13:22.455199 29821 catalog_manager.cc:4841] TS f364d0e1a3f14b55829f9ee56ac06792 (127.18.154.1:46809): tablet 385081bc5bff49af8acc777d47c5978f (table test-table [id=58a1faa89f99440db475570690d1dbb5]) successfully deleted
Aug 07 18:13:22 dist-test-slave-gkw7 krb5kdc[29758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126401, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.18.154.1@KRBTEST.COM
I20260807 18:13:22.466715 30822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:22.441745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60316 (local address 127.18.154.1:46809)
0807 18:13:22.451711 (+ 9966us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:22.451716 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:22.451737 (+ 21us) server_negotiation.cc:407] Connection header received
0807 18:13:22.451790 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:22.451796 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:22.451867 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:22.451973 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:22.453794 (+ 1821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:22.454791 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:22.456197 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:22.456601 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:22.456696 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:22.460658 (+ 3962us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:13:22.460675 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:13:22.460694 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:13:22.460726 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:13:22.463791 (+ 3065us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:22.464515 (+ 724us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:22.464523 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:22.464525 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:22.464717 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:22.465120 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:22.465124 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:22.465126 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:22.465490 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:13:22.465824 (+ 334us) server_negotiation.cc:1035] Waiting for connection context
0807 18:13:22.466151 (+ 327us) server_negotiation.cc:299] Negotiation successful
0807 18:13:22.466376 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9840,"thread_start_us":9687,"threads_started":1}
I20260807 18:13:22.467502 30152 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:13:22.568377 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 29953
I20260807 18:13:22.597093 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 30171
Aug 07 18:13:22 dist-test-slave-gkw7 krb5kdc[29758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126401, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.18.154.2@KRBTEST.COM
I20260807 18:13:22.614018 30824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:22.598052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58576 (local address 127.18.154.2:36977)
0807 18:13:22.598603 (+ 551us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:22.598608 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:22.598625 (+ 17us) server_negotiation.cc:407] Connection header received
0807 18:13:22.598681 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:22.598684 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:22.598745 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:22.598844 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:22.600263 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:22.601178 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:22.602529 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:22.602937 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:22.603033 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:22.607612 (+ 4579us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:13:22.607628 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:13:22.607647 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:13:22.607687 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:13:22.610875 (+ 3188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:22.611607 (+ 732us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:22.611617 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:22.611620 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:22.611767 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:22.612159 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:22.612162 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:22.612164 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:22.612611 (+ 447us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:13:22.612863 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0807 18:13:22.613340 (+ 477us) server_negotiation.cc:299] Negotiation successful
0807 18:13:22.613517 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":446,"thread_start_us":288,"threads_started":1}
I20260807 18:13:22.614871 30370 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:13:22.743713 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 30171
I20260807 18:13:22.776553 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 30383
Aug 07 18:13:22 dist-test-slave-gkw7 krb5kdc[29758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126401, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.18.154.3@KRBTEST.COM
I20260807 18:13:22.792130 30797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:22.777665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45238 (local address 127.18.154.3:37679)
0807 18:13:22.777841 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:22.777845 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:22.777862 (+ 17us) server_negotiation.cc:407] Connection header received
0807 18:13:22.777917 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:22.777920 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:22.777980 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:22.778062 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:22.779464 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:22.780206 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:22.781453 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:22.782002 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:22.782099 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:22.786498 (+ 4399us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:13:22.786514 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:13:22.786536 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:13:22.786585 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:13:22.789639 (+ 3054us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:22.790278 (+ 639us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:22.790286 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:22.790288 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:22.790427 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:22.790767 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:22.790771 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:22.790773 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:22.791146 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:13:22.791391 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0807 18:13:22.791674 (+ 283us) server_negotiation.cc:299] Negotiation successful
0807 18:13:22.791843 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 18:13:22.792822 30582 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:13:22.944494 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 30383
I20260807 18:13:22.978969 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 29788
I20260807 18:13:22.980255 29906 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:13:23.098731 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 29788
2026-08-07T18:13:23Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (7738 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786126328808105-19048-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:13:23 dist-test-slave-gkw7 krb5kdc[30832](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:13:23 dist-test-slave-gkw7 krb5kdc[30832](info): set up 2 sockets
Aug 07 18:13:23 dist-test-slave-gkw7 krb5kdc[30832](info): commencing operation
krb5kdc: starting...
W20260807 18:13:24.421023 19048 mini_kdc.cc:121] Time spent starting KDC: real 1.267s user 0.000s sys 0.033s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786126328808105-19048-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786126328808105-19048-0/krb5kdc/test-user.keytab.
Aug 07 18:13:24 dist-test-slave-gkw7 krb5kdc[30832](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126404, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:13:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:13:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.154.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.154.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.154.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.62.keytab.
Entry for principal kudu/127.18.154.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.62.keytab.
I20260807 18:13:24.699766 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786126328808105-19048-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786126328808105-19048-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786126328808105-19048-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.154.62:39643
--webserver_interface=127.18.154.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.62.keytab
--principal=kudu/127.18.154.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.154.20:39339
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.154.62:39643
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786126328808105-19048-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786126328808105-19048-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786126328808105-19048-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:13:25.048589 30854 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:13:25.048887 30854 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:13:25.048977 30854 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:13:25.056432 30854 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:13:25.056550 30854 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:13:25.056597 30854 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:13:25.056646 30854 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:13:25.056694 30854 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:13:25.066540 30854 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.154.20:39339
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786126328808105-19048-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786126328808105-19048-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.154.62:39643
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.62.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.154.62:39643
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.154.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786126328808105-19048-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:13:25.067822 30854 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:13:25.069082 30854 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:13:25.081746 30861 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:13:25.081789 30863 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:13:25.081677 30869 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:13:25.082121 30854 server_base.cc:1034] running on GCE node
I20260807 18:13:25.083133 30854 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:13:25.084623 30854 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:13:25.085875 30854 hybrid_clock.cc:648] HybridClock initialized: now 1786126405085803 us; error 57 us; skew 500 ppm
I20260807 18:13:25.086237 30854 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-07T18:13:25Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (2045 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:13:25 dist-test-slave-gkw7 krb5kdc[30876](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:13:25 dist-test-slave-gkw7 krb5kdc[30876](info): set up 2 sockets
Aug 07 18:13:25 dist-test-slave-gkw7 krb5kdc[30876](info): commencing operation
krb5kdc: starting...
W20260807 18:13:26.443522 19048 mini_kdc.cc:121] Time spent starting KDC: real 1.243s user 0.001s sys 0.034s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/krb5kdc/test-user.keytab.
Aug 07 18:13:26 dist-test-slave-gkw7 krb5kdc[30876](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126406, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:13:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:13:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.154.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.154.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.154.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.62.keytab.
Entry for principal kudu/127.18.154.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.62.keytab.
I20260807 18:13:26.723132 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.154.62:36685
--webserver_interface=127.18.154.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.62.keytab
--principal=kudu/127.18.154.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.154.20:34183
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.154.62:36685
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:13:27.069358 30896 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:13:27.069700 30896 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:13:27.069904 30896 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:13:27.077311 30896 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:13:27.077428 30896 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:13:27.077525 30896 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:13:27.077664 30896 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:13:27.077747 30896 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:13:27.087848 30896 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.154.20:34183
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.154.62:36685
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.62.keytab
--principal=kudu/127.18.154.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.154.62:36685
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.154.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:13:27.089347 30896 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:13:27.090734 30896 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:13:27.103379 30906 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:13:27.103487 30907 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:13:27.103439 30910 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:13:27.104210 30896 server_base.cc:1034] running on GCE node
I20260807 18:13:27.105062 30896 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:13:27.106755 30896 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:13:27.107982 30896 hybrid_clock.cc:648] HybridClock initialized: now 1786126407107931 us; error 36 us; skew 500 ppm
I20260807 18:13:27.108417 30896 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:13:27 dist-test-slave-gkw7 krb5kdc[30876](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126407, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:13:27.111954 30896 init.cc:377] Logged in from keytab as kudu/127.18.154.62@KRBTEST.COM (short username kudu)
I20260807 18:13:27.113822 30896 webserver.cc:460] Webserver started at http://127.18.154.62:37133/ using document root <none> and password file <none>
I20260807 18:13:27.114593 30896 fs_manager.cc:359] Metadata directory not provided
I20260807 18:13:27.114745 30896 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:13:27.115099 30896 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:13:27.118337 30896 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/minicluster-data/master-0/data/instance:
uuid: "7484a4cd74e84676971f7d7cd7bad337"
format_stamp: "Formatted at 2026-08-07 18:13:27 on dist-test-slave-gkw7"
server_key: "b1b35998ecf1aa83bbbd6d2fe02ea76f"
server_key_iv: "7f3e255112bcb5cdf0c421a809d6fe5e"
server_key_version: "encryptionkey@0"
I20260807 18:13:27.119155 30896 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/minicluster-data/master-0/wal/instance:
uuid: "7484a4cd74e84676971f7d7cd7bad337"
format_stamp: "Formatted at 2026-08-07 18:13:27 on dist-test-slave-gkw7"
server_key: "b1b35998ecf1aa83bbbd6d2fe02ea76f"
server_key_iv: "7f3e255112bcb5cdf0c421a809d6fe5e"
server_key_version: "encryptionkey@0"
I20260807 18:13:27.124800 30896 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260807 18:13:27.128737 30919 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:13:27.130494 30896 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 18:13:27.130709 30896 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/minicluster-data/master-0/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/minicluster-data/master-0/wal
uuid: "7484a4cd74e84676971f7d7cd7bad337"
format_stamp: "Formatted at 2026-08-07 18:13:27 on dist-test-slave-gkw7"
server_key: "b1b35998ecf1aa83bbbd6d2fe02ea76f"
server_key_iv: "7f3e255112bcb5cdf0c421a809d6fe5e"
server_key_version: "encryptionkey@0"
I20260807 18:13:27.130950 30896 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:13:27.144848 30896 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:13:27.149631 30896 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:13:27.149976 30896 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:13:27.171149 30995 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.154.62:36685 every 8 connection(s)
I20260807 18:13:27.171204 30896 rpc_server.cc:307] RPC server started. Bound to: 127.18.154.62:36685
I20260807 18:13:27.172742 30896 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
I20260807 18:13:27.176730 30997 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:13:27.181062 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 30896
I20260807 18:13:27.181316 19048 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/minicluster-data/master-0/wal/instance
I20260807 18:13:27.182190 19048 external_mini_cluster.cc:1428] Setting key 9b9973b2c6db80a991974705ca048d45
I20260807 18:13:27.190410 30997 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7484a4cd74e84676971f7d7cd7bad337: Bootstrap starting.
I20260807 18:13:27.194229 30997 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7484a4cd74e84676971f7d7cd7bad337: Neither blocks nor log segments found. Creating new log.
I20260807 18:13:27.195420 30997 log.cc:824] T 00000000000000000000000000000000 P 7484a4cd74e84676971f7d7cd7bad337: Log is configured to *not* fsync() on all Append() calls
I20260807 18:13:27.199174 30997 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7484a4cd74e84676971f7d7cd7bad337: No bootstrap required, opened a new log
Aug 07 18:13:27 dist-test-slave-gkw7 krb5kdc[30876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126406, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.154.62@KRBTEST.COM
I20260807 18:13:27.207329 30997 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7484a4cd74e84676971f7d7cd7bad337 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7484a4cd74e84676971f7d7cd7bad337" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 36685 } }
I20260807 18:13:27.207749 30997 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7484a4cd74e84676971f7d7cd7bad337 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:13:27.207897 30997 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7484a4cd74e84676971f7d7cd7bad337 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7484a4cd74e84676971f7d7cd7bad337, State: Initialized, Role: FOLLOWER
I20260807 18:13:27.208536 31002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:27.185670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38906 (local address 127.18.154.62:36685)
0807 18:13:27.187038 (+ 1368us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:27.187050 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:27.187098 (+ 48us) server_negotiation.cc:407] Connection header received
0807 18:13:27.188898 (+ 1800us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:27.188921 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:27.189313 (+ 392us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:27.189743 (+ 430us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:27.191605 (+ 1862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:27.192924 (+ 1319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:27.194795 (+ 1871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:27.195466 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:27.195680 (+ 214us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:27.200644 (+ 4964us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:13:27.200654 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:13:27.200669 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:13:27.200717 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:13:27.204149 (+ 3432us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:27.204883 (+ 734us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:27.204890 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:27.204896 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:27.205076 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:27.205490 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:27.205493 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:27.205495 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:27.206123 (+ 628us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:13:27.206486 (+ 363us) server_negotiation.cc:1035] Waiting for connection context
0807 18:13:27.206926 (+ 440us) server_negotiation.cc:299] Negotiation successful
0807 18:13:27.207362 (+ 436us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1079,"thread_start_us":729,"threads_started":1}
I20260807 18:13:27.208556 30997 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7484a4cd74e84676971f7d7cd7bad337 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7484a4cd74e84676971f7d7cd7bad337" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 36685 } }
I20260807 18:13:27.208945 30997 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7484a4cd74e84676971f7d7cd7bad337 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:13:27.209105 30997 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7484a4cd74e84676971f7d7cd7bad337 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:13:27.209313 30997 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7484a4cd74e84676971f7d7cd7bad337 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:13:27.212042 30997 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7484a4cd74e84676971f7d7cd7bad337 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7484a4cd74e84676971f7d7cd7bad337" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 36685 } }
I20260807 18:13:27.212477 30997 leader_election.cc:302] T 00000000000000000000000000000000 P 7484a4cd74e84676971f7d7cd7bad337 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7484a4cd74e84676971f7d7cd7bad337; no voters:
I20260807 18:13:27.213018 30997 leader_election.cc:288] T 00000000000000000000000000000000 P 7484a4cd74e84676971f7d7cd7bad337 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:13:27.213158 31007 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7484a4cd74e84676971f7d7cd7bad337 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:13:27.213888 31007 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7484a4cd74e84676971f7d7cd7bad337 [term 1 LEADER]: Becoming Leader. State: Replica: 7484a4cd74e84676971f7d7cd7bad337, State: Running, Role: LEADER
I20260807 18:13:27.214447 31007 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7484a4cd74e84676971f7d7cd7bad337 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7484a4cd74e84676971f7d7cd7bad337" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 36685 } }
I20260807 18:13:27.215169 30997 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7484a4cd74e84676971f7d7cd7bad337 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:13:27.217407 31008 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7484a4cd74e84676971f7d7cd7bad337 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7484a4cd74e84676971f7d7cd7bad337" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7484a4cd74e84676971f7d7cd7bad337" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 36685 } } }
I20260807 18:13:27.217813 31008 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7484a4cd74e84676971f7d7cd7bad337 [sys.catalog]: This master's current role is: LEADER
I20260807 18:13:27.217917 31009 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7484a4cd74e84676971f7d7cd7bad337 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7484a4cd74e84676971f7d7cd7bad337. Latest consensus state: current_term: 1 leader_uuid: "7484a4cd74e84676971f7d7cd7bad337" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7484a4cd74e84676971f7d7cd7bad337" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 36685 } } }
I20260807 18:13:27.218287 31009 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7484a4cd74e84676971f7d7cd7bad337 [sys.catalog]: This master's current role is: LEADER
I20260807 18:13:27.221555 31017 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:13:27.226397 31017 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:13:27.235699 31017 catalog_manager.cc:1347] Generated new cluster ID: 1cd6b30541c14244ac5249a71ee84002
I20260807 18:13:27.235824 31017 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:13:27.246429 31017 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:13:27.248528 31017 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:13:27.262552 31017 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7484a4cd74e84676971f7d7cd7bad337: Generated new TSK 0
I20260807 18:13:27.263705 31017 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.154.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.154.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.154.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.1.keytab.
Entry for principal kudu/127.18.154.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.1.keytab.
I20260807 18:13:27.333987 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.154.1:0
--local_ip_for_outbound_sockets=127.18.154.1
--webserver_interface=127.18.154.1
--webserver_port=0
--tserver_master_addrs=127.18.154.62:36685
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.1.keytab
--principal=kudu/127.18.154.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.154.20:34183
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:13:27.696969 31043 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:13:27.697381 31043 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:13:27.697614 31043 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:13:27.705374 31043 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:13:27.705490 31043 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:13:27.705713 31043 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.154.1
I20260807 18:13:27.716378 31043 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.154.20:34183
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.1.keytab
--principal=kudu/127.18.154.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.154.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.154.1
--webserver_port=0
--tserver_master_addrs=127.18.154.62:36685
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.154.1
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:13:27.717907 31043 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:13:27.719287 31043 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:13:27.733202 31050 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:13:27.733175 31051 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:13:27.733312 31056 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:13:27.733850 31043 server_base.cc:1034] running on GCE node
I20260807 18:13:27.734479 31043 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:13:27.735555 31043 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:13:27.736814 31043 hybrid_clock.cc:648] HybridClock initialized: now 1786126407736769 us; error 35 us; skew 500 ppm
I20260807 18:13:27.737277 31043 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:13:27 dist-test-slave-gkw7 krb5kdc[30876](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126407, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:13:27.740708 31043 init.cc:377] Logged in from keytab as kudu/127.18.154.1@KRBTEST.COM (short username kudu)
I20260807 18:13:27.742403 31043 webserver.cc:460] Webserver started at http://127.18.154.1:38283/ using document root <none> and password file <none>
I20260807 18:13:27.743150 31043 fs_manager.cc:359] Metadata directory not provided
I20260807 18:13:27.743285 31043 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:13:27.743633 31043 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:13:27.746745 31043 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/minicluster-data/ts-0/data/instance:
uuid: "b1e0081e65e342618eaa54f2fa606987"
format_stamp: "Formatted at 2026-08-07 18:13:27 on dist-test-slave-gkw7"
server_key: "3922910fd0d75f1055d9e0d93bbee67c"
server_key_iv: "7cfdcbd29b253b9f34de6a34c9d81422"
server_key_version: "encryptionkey@0"
I20260807 18:13:27.747547 31043 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/minicluster-data/ts-0/wal/instance:
uuid: "b1e0081e65e342618eaa54f2fa606987"
format_stamp: "Formatted at 2026-08-07 18:13:27 on dist-test-slave-gkw7"
server_key: "3922910fd0d75f1055d9e0d93bbee67c"
server_key_iv: "7cfdcbd29b253b9f34de6a34c9d81422"
server_key_version: "encryptionkey@0"
I20260807 18:13:27.752781 31043 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 18:13:27.756397 31064 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:13:27.757848 31043 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:13:27.758018 31043 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/minicluster-data/ts-0/wal
uuid: "b1e0081e65e342618eaa54f2fa606987"
format_stamp: "Formatted at 2026-08-07 18:13:27 on dist-test-slave-gkw7"
server_key: "3922910fd0d75f1055d9e0d93bbee67c"
server_key_iv: "7cfdcbd29b253b9f34de6a34c9d81422"
server_key_version: "encryptionkey@0"
I20260807 18:13:27.758226 31043 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:13:27.795184 31043 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:13:27.799935 31043 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:13:27.800320 31043 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:13:27.801393 31043 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:13:27.802917 31043 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:13:27.803025 31043 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:13:27.803188 31043 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:13:27.803272 31043 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:13:27.839085 31043 rpc_server.cc:307] RPC server started. Bound to: 127.18.154.1:44115
I20260807 18:13:27.839130 31245 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.154.1:44115 every 8 connection(s)
I20260807 18:13:27.840468 31043 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/minicluster-data/ts-0/data/info.pb
I20260807 18:13:27.842957 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 31043
I20260807 18:13:27.843196 19048 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/minicluster-data/ts-0/wal/instance
I20260807 18:13:27.844028 19048 external_mini_cluster.cc:1428] Setting key 1308bb25fafd753a7ff3caf31194cc56
Aug 07 18:13:27 dist-test-slave-gkw7 krb5kdc[30876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126407, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.1@KRBTEST.COM for kudu/127.18.154.62@KRBTEST.COM
I20260807 18:13:27.863536 31249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:27.843441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.1:52507 (local address 127.18.154.62:36685)
0807 18:13:27.843939 (+ 498us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:27.843943 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:27.845375 (+ 1432us) server_negotiation.cc:407] Connection header received
0807 18:13:27.847606 (+ 2231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:27.847614 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:27.847726 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:27.847856 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:27.849438 (+ 1582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:27.850625 (+ 1187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:27.852262 (+ 1637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:27.852980 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:27.853147 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:27.857037 (+ 3890us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:13:27.857044 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:13:27.857049 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:13:27.857095 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:13:27.859875 (+ 2780us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:27.860848 (+ 973us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:27.860851 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:27.860854 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:27.860974 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:27.861500 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:27.861503 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:27.861505 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:27.861918 (+ 413us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:13:27.862034 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0807 18:13:27.863074 (+ 1040us) server_negotiation.cc:299] Negotiation successful
0807 18:13:27.863263 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":387,"thread_start_us":273,"threads_started":1}
I20260807 18:13:27.865360 31251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:27.844201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:36685 (local address 127.18.154.1:52507)
0807 18:13:27.845182 (+ 981us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:27.845231 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0807 18:13:27.847308 (+ 2077us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:13:27.848088 (+ 780us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:13:27.848097 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:13:27.848634 (+ 537us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:13:27.849249 (+ 615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:27.849267 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:27.850791 (+ 1524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:27.850795 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:27.852096 (+ 1301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:27.852109 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:27.853230 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:27.853236 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:27.853443 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:27.854278 (+ 835us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:13:27.854307 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:13:27.856785 (+ 2478us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:13:27.860026 (+ 3241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:13:27.860039 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:13:27.860060 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:13:27.860728 (+ 668us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:13:27.861112 (+ 384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:13:27.861119 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:13:27.861126 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:13:27.861384 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:13:27.862049 (+ 665us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:13:27.862060 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:13:27.862541 (+ 481us) client_negotiation.cc:769] Sending connection context
0807 18:13:27.862969 (+ 428us) client_negotiation.cc:240] Negotiation successful
0807 18:13:27.863594 (+ 625us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":653,"thread_start_us":405,"threads_started":1}
I20260807 18:13:27.867097 31246 heartbeater.cc:347] Connected to a master server at 127.18.154.62:36685
I20260807 18:13:27.867468 31246 heartbeater.cc:464] Registering TS with master...
I20260807 18:13:27.868257 31246 heartbeater.cc:511] Master 127.18.154.62:36685 requested a full tablet report, sending...
I20260807 18:13:27.870498 30943 ts_manager.cc:194] Registered new tserver with Master: b1e0081e65e342618eaa54f2fa606987 (127.18.154.1:44115)
I20260807 18:13:27.871992 30943 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.154.1@KRBTEST.COM'} at 127.18.154.1:52507
WARNING: no policy specified for kudu/127.18.154.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.154.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.154.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.2.keytab.
Entry for principal kudu/127.18.154.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.2.keytab.
I20260807 18:13:27.909489 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.154.2:0
--local_ip_for_outbound_sockets=127.18.154.2
--webserver_interface=127.18.154.2
--webserver_port=0
--tserver_master_addrs=127.18.154.62:36685
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.2.keytab
--principal=kudu/127.18.154.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.154.20:34183
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:13:28.259500 31254 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:13:28.259793 31254 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:13:28.259888 31254 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:13:28.267436 31254 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:13:28.267552 31254 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:13:28.267666 31254 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.154.2
I20260807 18:13:28.277671 31254 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.154.20:34183
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.2.keytab
--principal=kudu/127.18.154.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.154.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.154.2
--webserver_port=0
--tserver_master_addrs=127.18.154.62:36685
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.154.2
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:13:28.278962 31254 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:13:28.280263 31254 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:13:28.293874 31265 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:13:28.293919 31263 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:13:28.293952 31262 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:13:28.294708 31254 server_base.cc:1034] running on GCE node
I20260807 18:13:28.295390 31254 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:13:28.296425 31254 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:13:28.297662 31254 hybrid_clock.cc:648] HybridClock initialized: now 1786126408297609 us; error 41 us; skew 500 ppm
I20260807 18:13:28.298038 31254 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:13:28 dist-test-slave-gkw7 krb5kdc[30876](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126408, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:13:28.305961 31254 init.cc:377] Logged in from keytab as kudu/127.18.154.2@KRBTEST.COM (short username kudu)
I20260807 18:13:28.307605 31254 webserver.cc:460] Webserver started at http://127.18.154.2:42043/ using document root <none> and password file <none>
I20260807 18:13:28.308357 31254 fs_manager.cc:359] Metadata directory not provided
I20260807 18:13:28.308507 31254 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:13:28.308858 31254 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:13:28.311967 31254 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/minicluster-data/ts-1/data/instance:
uuid: "1f58e5c57f8d462dbb14b49d50af9c35"
format_stamp: "Formatted at 2026-08-07 18:13:28 on dist-test-slave-gkw7"
server_key: "99c487560e3a3c164d230e694de3bd7f"
server_key_iv: "0ae4cd0df78c383c3b6b6bc464a45a7e"
server_key_version: "encryptionkey@0"
I20260807 18:13:28.312791 31254 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/minicluster-data/ts-1/wal/instance:
uuid: "1f58e5c57f8d462dbb14b49d50af9c35"
format_stamp: "Formatted at 2026-08-07 18:13:28 on dist-test-slave-gkw7"
server_key: "99c487560e3a3c164d230e694de3bd7f"
server_key_iv: "0ae4cd0df78c383c3b6b6bc464a45a7e"
server_key_version: "encryptionkey@0"
I20260807 18:13:28.319028 31254 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.002s
I20260807 18:13:28.322934 31274 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:13:28.324548 31254 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 18:13:28.324756 31254 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/minicluster-data/ts-1/wal
uuid: "1f58e5c57f8d462dbb14b49d50af9c35"
format_stamp: "Formatted at 2026-08-07 18:13:28 on dist-test-slave-gkw7"
server_key: "99c487560e3a3c164d230e694de3bd7f"
server_key_iv: "0ae4cd0df78c383c3b6b6bc464a45a7e"
server_key_version: "encryptionkey@0"
I20260807 18:13:28.324996 31254 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:13:28.367167 31254 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:13:28.371829 31254 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:13:28.372179 31254 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:13:28.373241 31254 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:13:28.374774 31254 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:13:28.374883 31254 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:13:28.375034 31254 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:13:28.375109 31254 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:13:28.410977 31254 rpc_server.cc:307] RPC server started. Bound to: 127.18.154.2:36957
I20260807 18:13:28.411036 31433 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.154.2:36957 every 8 connection(s)
I20260807 18:13:28.412348 31254 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/minicluster-data/ts-1/data/info.pb
I20260807 18:13:28.419746 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 31254
I20260807 18:13:28.419986 19048 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/minicluster-data/ts-1/wal/instance
I20260807 18:13:28.420476 19048 external_mini_cluster.cc:1428] Setting key b3eead7c2410163c6709244367c99755
Aug 07 18:13:28 dist-test-slave-gkw7 krb5kdc[30876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126408, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.2@KRBTEST.COM for kudu/127.18.154.62@KRBTEST.COM
I20260807 18:13:28.436044 31437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:28.415221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.2:52813 (local address 127.18.154.62:36685)
0807 18:13:28.415693 (+ 472us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:28.415699 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:28.416867 (+ 1168us) server_negotiation.cc:407] Connection header received
0807 18:13:28.418871 (+ 2004us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:28.418876 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:28.418947 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:28.419050 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:28.421074 (+ 2024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:28.421984 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:28.423820 (+ 1836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:28.424649 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:28.424811 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:28.428572 (+ 3761us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:13:28.428581 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:13:28.428589 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:13:28.428648 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:13:28.432607 (+ 3959us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:28.433441 (+ 834us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:28.433448 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:28.433451 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:28.433670 (+ 219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:28.434124 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:28.434129 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:28.434133 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:28.434424 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:13:28.434582 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0807 18:13:28.435362 (+ 780us) server_negotiation.cc:299] Negotiation successful
0807 18:13:28.435650 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":341,"thread_start_us":215,"threads_started":1}
I20260807 18:13:28.437110 31439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:28.415922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:36685 (local address 127.18.154.2:52813)
0807 18:13:28.416700 (+ 778us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:28.416740 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 18:13:28.418604 (+ 1864us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:13:28.419270 (+ 666us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:13:28.419278 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:13:28.420200 (+ 922us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:13:28.420878 (+ 678us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:28.420899 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:28.422199 (+ 1300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:28.422204 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:28.423650 (+ 1446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:28.423664 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:28.424893 (+ 1229us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:28.424899 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:28.425134 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:28.425948 (+ 814us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:13:28.425973 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:13:28.428303 (+ 2330us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:13:28.432788 (+ 4485us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:13:28.432796 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:13:28.432811 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:13:28.433296 (+ 485us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:13:28.433795 (+ 499us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:13:28.433806 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:13:28.433809 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:13:28.434001 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:13:28.434595 (+ 594us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:13:28.434606 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:13:28.434970 (+ 364us) client_negotiation.cc:769] Sending connection context
0807 18:13:28.435261 (+ 291us) client_negotiation.cc:240] Negotiation successful
0807 18:13:28.435863 (+ 602us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":521,"thread_start_us":327,"threads_started":1}
I20260807 18:13:28.438746 31434 heartbeater.cc:347] Connected to a master server at 127.18.154.62:36685
I20260807 18:13:28.439090 31434 heartbeater.cc:464] Registering TS with master...
I20260807 18:13:28.439868 31434 heartbeater.cc:511] Master 127.18.154.62:36685 requested a full tablet report, sending...
I20260807 18:13:28.441414 30943 ts_manager.cc:194] Registered new tserver with Master: 1f58e5c57f8d462dbb14b49d50af9c35 (127.18.154.2:36957)
I20260807 18:13:28.442292 30943 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.154.2@KRBTEST.COM'} at 127.18.154.2:52813
WARNING: no policy specified for kudu/127.18.154.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.154.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.154.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.3.keytab.
Entry for principal kudu/127.18.154.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.3.keytab.
I20260807 18:13:28.484516 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.154.3:0
--local_ip_for_outbound_sockets=127.18.154.3
--webserver_interface=127.18.154.3
--webserver_port=0
--tserver_master_addrs=127.18.154.62:36685
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.3.keytab
--principal=kudu/127.18.154.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.154.20:34183
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:13:28.830057 31443 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:13:28.830413 31443 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:13:28.830602 31443 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:13:28.838078 31443 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:13:28.838181 31443 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:13:28.838392 31443 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.154.3
I20260807 18:13:28.848372 31443 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.154.20:34183
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.3.keytab
--principal=kudu/127.18.154.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.154.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.154.3
--webserver_port=0
--tserver_master_addrs=127.18.154.62:36685
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.154.3
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:13:28.849730 31443 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:13:28.851099 31443 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:13:28.864603 31449 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:13:28.864686 31452 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:13:28.864533 31455 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:13:28.865697 31443 server_base.cc:1034] running on GCE node
I20260807 18:13:28.866385 31443 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:13:28.867339 31443 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:13:28.868546 31443 hybrid_clock.cc:648] HybridClock initialized: now 1786126408868477 us; error 57 us; skew 500 ppm
I20260807 18:13:28.868872 31443 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:13:28 dist-test-slave-gkw7 krb5kdc[30876](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126408, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:13:28.872342 31443 init.cc:377] Logged in from keytab as kudu/127.18.154.3@KRBTEST.COM (short username kudu)
I20260807 18:13:28.873994 31443 webserver.cc:460] Webserver started at http://127.18.154.3:36917/ using document root <none> and password file <none>
I20260807 18:13:28.874717 31443 fs_manager.cc:359] Metadata directory not provided
I20260807 18:13:28.874859 31443 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:13:28.875216 31443 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:13:28.875205 31246 heartbeater.cc:503] Master 127.18.154.62:36685 was elected leader, sending a full tablet report...
I20260807 18:13:28.878479 31443 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/minicluster-data/ts-2/data/instance:
uuid: "c2f6d5d122154c208c979ecaeab97274"
format_stamp: "Formatted at 2026-08-07 18:13:28 on dist-test-slave-gkw7"
server_key: "9fec385ce423619b5818cab37e153a16"
server_key_iv: "afabb28a8b5005ca10ce61356be8c64b"
server_key_version: "encryptionkey@0"
I20260807 18:13:28.879278 31443 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/minicluster-data/ts-2/wal/instance:
uuid: "c2f6d5d122154c208c979ecaeab97274"
format_stamp: "Formatted at 2026-08-07 18:13:28 on dist-test-slave-gkw7"
server_key: "9fec385ce423619b5818cab37e153a16"
server_key_iv: "afabb28a8b5005ca10ce61356be8c64b"
server_key_version: "encryptionkey@0"
I20260807 18:13:28.884532 31443 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 18:13:28.888052 31462 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:13:28.889443 31443 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 18:13:28.889670 31443 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/minicluster-data/ts-2/wal
uuid: "c2f6d5d122154c208c979ecaeab97274"
format_stamp: "Formatted at 2026-08-07 18:13:28 on dist-test-slave-gkw7"
server_key: "9fec385ce423619b5818cab37e153a16"
server_key_iv: "afabb28a8b5005ca10ce61356be8c64b"
server_key_version: "encryptionkey@0"
I20260807 18:13:28.889883 31443 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:13:28.912452 31443 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:13:28.916934 31443 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:13:28.917241 31443 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:13:28.918304 31443 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:13:28.919641 31443 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:13:28.919756 31443 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:13:28.919926 31443 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:13:28.919977 31443 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:13:28.954679 31443 rpc_server.cc:307] RPC server started. Bound to: 127.18.154.3:34321
I20260807 18:13:28.954774 31628 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.154.3:34321 every 8 connection(s)
I20260807 18:13:28.956054 31443 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/minicluster-data/ts-2/data/info.pb
I20260807 18:13:28.963020 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 31443
I20260807 18:13:28.963244 19048 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/minicluster-data/ts-2/wal/instance
I20260807 18:13:28.963850 19048 external_mini_cluster.cc:1428] Setting key b5c61276ce094bb17232e099543f103c
Aug 07 18:13:28 dist-test-slave-gkw7 krb5kdc[30876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126408, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.3@KRBTEST.COM for kudu/127.18.154.62@KRBTEST.COM
I20260807 18:13:28.978058 31631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:28.958900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.3:34615 (local address 127.18.154.62:36685)
0807 18:13:28.959435 (+ 535us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:28.959442 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:28.960530 (+ 1088us) server_negotiation.cc:407] Connection header received
0807 18:13:28.962755 (+ 2225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:28.962760 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:28.962829 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:28.962927 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:28.965212 (+ 2285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:28.966330 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:28.968292 (+ 1962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:28.968665 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:28.968771 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:28.972493 (+ 3722us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:13:28.972498 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:13:28.972501 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:13:28.972533 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:13:28.975077 (+ 2544us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:28.975774 (+ 697us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:28.975777 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:28.975780 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:28.975892 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:28.976328 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:28.976331 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:28.976332 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:28.976576 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:13:28.976682 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0807 18:13:28.977428 (+ 746us) server_negotiation.cc:299] Negotiation successful
0807 18:13:28.977651 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":399,"thread_start_us":243,"threads_started":1}
I20260807 18:13:28.979408 31632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:28.959455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:36685 (local address 127.18.154.3:34615)
0807 18:13:28.960357 (+ 902us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:28.960398 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 18:13:28.962410 (+ 2012us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:13:28.963273 (+ 863us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:13:28.963287 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:13:28.964140 (+ 853us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:13:28.965015 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:28.965048 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:28.966494 (+ 1446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:28.966498 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:28.968160 (+ 1662us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:28.968173 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:28.968868 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:28.968873 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:28.969081 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:28.969935 (+ 854us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:13:28.969960 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:13:28.972286 (+ 2326us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:13:28.975207 (+ 2921us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:13:28.975214 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:13:28.975228 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:13:28.975673 (+ 445us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:13:28.975993 (+ 320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:13:28.975996 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:13:28.975999 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:13:28.976231 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:13:28.976692 (+ 461us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:13:28.976698 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:13:28.977013 (+ 315us) client_negotiation.cc:769] Sending connection context
0807 18:13:28.977347 (+ 334us) client_negotiation.cc:240] Negotiation successful
0807 18:13:28.977941 (+ 594us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":512,"thread_start_us":287,"threads_started":1}
I20260807 18:13:28.980880 31629 heartbeater.cc:347] Connected to a master server at 127.18.154.62:36685
I20260807 18:13:28.981220 31629 heartbeater.cc:464] Registering TS with master...
I20260807 18:13:28.982061 31629 heartbeater.cc:511] Master 127.18.154.62:36685 requested a full tablet report, sending...
I20260807 18:13:28.983526 30943 ts_manager.cc:194] Registered new tserver with Master: c2f6d5d122154c208c979ecaeab97274 (127.18.154.3:34321)
I20260807 18:13:28.984279 30943 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.154.3@KRBTEST.COM'} at 127.18.154.3:34615
I20260807 18:13:28.991881 19048 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.18.154.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.154.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.154.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/krb5kdc/oryx_127.18.154.1.keytab.
Entry for principal oryx/127.18.154.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/krb5kdc/oryx_127.18.154.1.keytab.
WARNING: no policy specified for oryx/127.18.154.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.154.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.154.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/krb5kdc/oryx_127.18.154.2.keytab.
Entry for principal oryx/127.18.154.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/krb5kdc/oryx_127.18.154.2.keytab.
WARNING: no policy specified for oryx/127.18.154.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.154.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.154.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/krb5kdc/oryx_127.18.154.3.keytab.
Entry for principal oryx/127.18.154.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/krb5kdc/oryx_127.18.154.3.keytab.
I20260807 18:13:29.167872 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 31043
Aug 07 18:13:29 dist-test-slave-gkw7 krb5kdc[30876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126406, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.154.1@KRBTEST.COM
I20260807 18:13:29.186913 31642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:29.169171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38354 (local address 127.18.154.1:44115)
0807 18:13:29.169876 (+ 705us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:29.169884 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:29.169915 (+ 31us) server_negotiation.cc:407] Connection header received
0807 18:13:29.169979 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:29.169985 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:29.170226 (+ 241us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:29.170367 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:29.172234 (+ 1867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:29.173269 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:29.175155 (+ 1886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:29.175932 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:29.176038 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:29.180832 (+ 4794us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:13:29.180844 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:13:29.180859 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:13:29.180900 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:13:29.183957 (+ 3057us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:29.184688 (+ 731us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:29.184695 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:29.184698 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:29.184828 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:29.185226 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:29.185230 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:29.185233 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:29.185651 (+ 418us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:13:29.185989 (+ 338us) server_negotiation.cc:1035] Waiting for connection context
0807 18:13:29.186298 (+ 309us) server_negotiation.cc:299] Negotiation successful
0807 18:13:29.186570 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":556,"thread_start_us":404,"threads_started":1}
I20260807 18:13:29.188251 31239 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:13:29.273018 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 31043
I20260807 18:13:29.295348 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 31254
Aug 07 18:13:29 dist-test-slave-gkw7 krb5kdc[30876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126406, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.154.2@KRBTEST.COM
I20260807 18:13:29.311627 31645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:29.296287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60380 (local address 127.18.154.2:36957)
0807 18:13:29.296896 (+ 609us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:29.296903 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:29.296930 (+ 27us) server_negotiation.cc:407] Connection header received
0807 18:13:29.296991 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:29.296997 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:29.297215 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:29.297348 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:29.298667 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:29.299720 (+ 1053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:29.300963 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:29.301512 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:29.301700 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:29.306230 (+ 4530us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:13:29.306242 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:13:29.306258 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:13:29.306292 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:13:29.309002 (+ 2710us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:29.309710 (+ 708us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:29.309722 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:29.309730 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:29.309853 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:29.310185 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:29.310188 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:29.310190 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:29.310522 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:13:29.310770 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0807 18:13:29.310995 (+ 225us) server_negotiation.cc:299] Negotiation successful
0807 18:13:29.311200 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":467,"thread_start_us":313,"threads_started":1}
I20260807 18:13:29.312963 31427 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:13:29.396209 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 31254
I20260807 18:13:29.418007 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 31443
Aug 07 18:13:29 dist-test-slave-gkw7 krb5kdc[30876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126406, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.154.3@KRBTEST.COM
I20260807 18:13:29.435155 31647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:29.419082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60918 (local address 127.18.154.3:34321)
0807 18:13:29.419636 (+ 554us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:29.419644 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:29.419675 (+ 31us) server_negotiation.cc:407] Connection header received
0807 18:13:29.419740 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:29.419746 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:29.419967 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:29.420157 (+ 190us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:29.421633 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:29.422658 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:29.423882 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:29.424452 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:29.424550 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:29.429060 (+ 4510us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:13:29.429073 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:13:29.429089 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:13:29.429131 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:13:29.432340 (+ 3209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:29.433020 (+ 680us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:29.433028 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:29.433030 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:29.433158 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:29.433501 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:29.433504 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:29.433506 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:29.433923 (+ 417us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:13:29.434147 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0807 18:13:29.434475 (+ 328us) server_negotiation.cc:299] Negotiation successful
0807 18:13:29.434687 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":402,"thread_start_us":253,"threads_started":1}
I20260807 18:13:29.436597 31624 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:13:29.522615 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 31443
I20260807 18:13:29.547194 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 30896
I20260807 18:13:29.548589 30990 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:13:29.653661 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 30896
I20260807 18:13:29.681452 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.154.62:36685
--webserver_interface=127.18.154.62
--webserver_port=37133
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.62.keytab
--principal=kudu/127.18.154.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.154.20:34183
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.154.62:36685
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:13:30.024847 31653 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:13:30.025153 31653 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:13:30.025246 31653 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:13:30.033121 31653 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:13:30.033232 31653 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:13:30.033286 31653 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:13:30.033344 31653 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:13:30.033388 31653 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:13:30.043447 31653 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.154.20:34183
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.154.62:36685
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.62.keytab
--principal=kudu/127.18.154.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.154.62:36685
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.154.62
--webserver_port=37133
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:13:30.044759 31653 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:13:30.046083 31653 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:13:30.058780 31660 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:13:30.058884 31662 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:13:30.059046 31664 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:13:30.060609 31653 server_base.cc:1034] running on GCE node
I20260807 18:13:30.061415 31653 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:13:30.063191 31653 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:13:30.064419 31653 hybrid_clock.cc:648] HybridClock initialized: now 1786126410064370 us; error 31 us; skew 500 ppm
I20260807 18:13:30.064805 31653 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:13:30 dist-test-slave-gkw7 krb5kdc[30876](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126410, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:13:30.068173 31653 init.cc:377] Logged in from keytab as kudu/127.18.154.62@KRBTEST.COM (short username kudu)
I20260807 18:13:30.069728 31653 webserver.cc:460] Webserver started at http://127.18.154.62:37133/ using document root <none> and password file <none>
I20260807 18:13:30.070497 31653 fs_manager.cc:359] Metadata directory not provided
I20260807 18:13:30.070650 31653 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:13:30.076323 31653 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.004s
I20260807 18:13:30.079360 31678 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:13:30.080785 31653 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 18:13:30.080976 31653 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/minicluster-data/master-0/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/minicluster-data/master-0/wal
uuid: "7484a4cd74e84676971f7d7cd7bad337"
format_stamp: "Formatted at 2026-08-07 18:13:27 on dist-test-slave-gkw7"
server_key: "b1b35998ecf1aa83bbbd6d2fe02ea76f"
server_key_iv: "7f3e255112bcb5cdf0c421a809d6fe5e"
server_key_version: "encryptionkey@0"
I20260807 18:13:30.082090 31653 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:13:30.107375 31653 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:13:30.112159 31653 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:13:30.112396 31653 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:13:30.131937 31653 rpc_server.cc:307] RPC server started. Bound to: 127.18.154.62:36685
I20260807 18:13:30.131940 31757 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.154.62:36685 every 8 connection(s)
I20260807 18:13:30.133569 31653 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
I20260807 18:13:30.138797 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 31653
I20260807 18:13:30.139617 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.154.1:44115
--local_ip_for_outbound_sockets=127.18.154.1
--tserver_master_addrs=127.18.154.62:36685
--webserver_port=38283
--webserver_interface=127.18.154.1
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.1.keytab
--principal=kudu/127.18.154.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.154.20:34183
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.18.154.1
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/krb5kdc/oryx_127.18.154.1.keytab with env {KRB5CCNAME=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 18:13:30.140942 31758 sys_catalog.cc:263] Verifying existing consensus state
I20260807 18:13:30.144305 31758 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7484a4cd74e84676971f7d7cd7bad337: Bootstrap starting.
I20260807 18:13:30.163534 31758 log.cc:824] T 00000000000000000000000000000000 P 7484a4cd74e84676971f7d7cd7bad337: Log is configured to *not* fsync() on all Append() calls
I20260807 18:13:30.172528 31758 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7484a4cd74e84676971f7d7cd7bad337: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 18:13:30.173076 31758 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7484a4cd74e84676971f7d7cd7bad337: Bootstrap complete.
I20260807 18:13:30.182302 31758 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7484a4cd74e84676971f7d7cd7bad337 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7484a4cd74e84676971f7d7cd7bad337" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 36685 } }
I20260807 18:13:30.183200 31758 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7484a4cd74e84676971f7d7cd7bad337 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7484a4cd74e84676971f7d7cd7bad337, State: Initialized, Role: FOLLOWER
I20260807 18:13:30.183887 31758 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7484a4cd74e84676971f7d7cd7bad337 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7484a4cd74e84676971f7d7cd7bad337" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 36685 } }
I20260807 18:13:30.184156 31758 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7484a4cd74e84676971f7d7cd7bad337 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:13:30.184285 31758 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7484a4cd74e84676971f7d7cd7bad337 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:13:30.184479 31758 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7484a4cd74e84676971f7d7cd7bad337 [term 1 FOLLOWER]: Advancing to term 2
I20260807 18:13:30.188588 31758 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7484a4cd74e84676971f7d7cd7bad337 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7484a4cd74e84676971f7d7cd7bad337" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 36685 } }
I20260807 18:13:30.189080 31758 leader_election.cc:302] T 00000000000000000000000000000000 P 7484a4cd74e84676971f7d7cd7bad337 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7484a4cd74e84676971f7d7cd7bad337; no voters:
I20260807 18:13:30.189741 31758 leader_election.cc:288] T 00000000000000000000000000000000 P 7484a4cd74e84676971f7d7cd7bad337 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 18:13:30.189877 31763 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7484a4cd74e84676971f7d7cd7bad337 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 18:13:30.191058 31763 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7484a4cd74e84676971f7d7cd7bad337 [term 2 LEADER]: Becoming Leader. State: Replica: 7484a4cd74e84676971f7d7cd7bad337, State: Running, Role: LEADER
I20260807 18:13:30.191601 31763 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7484a4cd74e84676971f7d7cd7bad337 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7484a4cd74e84676971f7d7cd7bad337" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 36685 } }
I20260807 18:13:30.192214 31758 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7484a4cd74e84676971f7d7cd7bad337 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:13:30.194190 31766 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7484a4cd74e84676971f7d7cd7bad337 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7484a4cd74e84676971f7d7cd7bad337. Latest consensus state: current_term: 2 leader_uuid: "7484a4cd74e84676971f7d7cd7bad337" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7484a4cd74e84676971f7d7cd7bad337" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 36685 } } }
I20260807 18:13:30.194609 31766 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7484a4cd74e84676971f7d7cd7bad337 [sys.catalog]: This master's current role is: LEADER
I20260807 18:13:30.195144 31765 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7484a4cd74e84676971f7d7cd7bad337 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "7484a4cd74e84676971f7d7cd7bad337" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7484a4cd74e84676971f7d7cd7bad337" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 36685 } } }
I20260807 18:13:30.195526 31765 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7484a4cd74e84676971f7d7cd7bad337 [sys.catalog]: This master's current role is: LEADER
I20260807 18:13:30.206719 31773 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:13:30.213965 31773 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:13:30.215787 31773 catalog_manager.cc:1259] Loaded cluster ID: 1cd6b30541c14244ac5249a71ee84002
I20260807 18:13:30.215905 31773 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:13:30.221742 31773 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:13:30.224331 31773 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 7484a4cd74e84676971f7d7cd7bad337: Loaded TSK: 0
I20260807 18:13:30.226269 31773 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 18:13:30.512583 31760 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:13:30.512858 31760 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:13:30.513031 31760 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:13:30.520627 31760 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:13:30.520753 31760 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:13:30.520926 31760 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.154.1
I20260807 18:13:30.531296 31760 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.154.20:34183
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/krb5kdc/oryx_127.18.154.1.keytab
--principal=oryx/127.18.154.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.154.1:44115
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.154.1
--webserver_port=38283
--tserver_master_addrs=127.18.154.62:36685
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.154.1
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:13:30.532706 31760 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:13:30.534107 31760 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:13:30.548302 31798 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:13:30.549022 31799 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:13:30.550323 31803 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:13:30.551100 31760 server_base.cc:1034] running on GCE node
I20260807 18:13:30.551975 31760 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:13:30.553365 31760 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:13:30.554656 31760 hybrid_clock.cc:648] HybridClock initialized: now 1786126410554576 us; error 78 us; skew 500 ppm
I20260807 18:13:30.555073 31760 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:13:30 dist-test-slave-gkw7 krb5kdc[30876](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126410, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.154.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:13:30.559430 31760 init.cc:377] Logged in from keytab as oryx/127.18.154.1@KRBTEST.COM (short username oryx)
I20260807 18:13:30.561223 31760 webserver.cc:460] Webserver started at http://127.18.154.1:38283/ using document root <none> and password file <none>
I20260807 18:13:30.562156 31760 fs_manager.cc:359] Metadata directory not provided
I20260807 18:13:30.562342 31760 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:13:30.568990 31760 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 18:13:30.572140 31816 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:13:30.573814 31760 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:13:30.574014 31760 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/minicluster-data/ts-0/wal
uuid: "b1e0081e65e342618eaa54f2fa606987"
format_stamp: "Formatted at 2026-08-07 18:13:27 on dist-test-slave-gkw7"
server_key: "3922910fd0d75f1055d9e0d93bbee67c"
server_key_iv: "7cfdcbd29b253b9f34de6a34c9d81422"
server_key_version: "encryptionkey@0"
I20260807 18:13:30.575094 31760 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:13:30.591240 31760 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:13:30.596663 31760 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:13:30.596957 31760 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:13:30.598114 31760 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:13:30.600040 31760 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:13:30.600165 31760 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:13:30.600281 31760 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:13:30.600355 31760 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:13:30.635485 31760 rpc_server.cc:307] RPC server started. Bound to: 127.18.154.1:44115
I20260807 18:13:30.635586 31984 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.154.1:44115 every 8 connection(s)
I20260807 18:13:30.637079 31760 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/minicluster-data/ts-0/data/info.pb
I20260807 18:13:30.638696 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 31760
I20260807 18:13:30.639472 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.154.2:36957
--local_ip_for_outbound_sockets=127.18.154.2
--tserver_master_addrs=127.18.154.62:36685
--webserver_port=42043
--webserver_interface=127.18.154.2
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.2.keytab
--principal=kudu/127.18.154.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.154.20:34183
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.18.154.2
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/krb5kdc/oryx_127.18.154.2.keytab with env {KRB5CCNAME=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 18:13:30 dist-test-slave-gkw7 krb5kdc[30876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.154.1@KRBTEST.COM for oryx/127.18.154.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:13:30 dist-test-slave-gkw7 krb5kdc[30876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.154.1@KRBTEST.COM for oryx/127.18.154.62@KRBTEST.COM, Server not found in Kerberos database
W20260807 18:13:30.991695 31995 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:13:30.992015 31995 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:13:30.992177 31995 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:13:30.999681 31995 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:13:30.999802 31995 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:13:31.000018 31995 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.154.2
I20260807 18:13:31.009912 31995 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.154.20:34183
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/krb5kdc/oryx_127.18.154.2.keytab
--principal=oryx/127.18.154.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.154.2:36957
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.154.2
--webserver_port=42043
--tserver_master_addrs=127.18.154.62:36685
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.154.2
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:13:31.011284 31995 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:13:31.012643 31995 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:13:31.031129 32002 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:13:31.031083 32004 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:13:31.032085 32006 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:13:31.032565 31995 server_base.cc:1034] running on GCE node
I20260807 18:13:31.033272 31995 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:13:31.034451 31995 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:13:31.035686 31995 hybrid_clock.cc:648] HybridClock initialized: now 1786126411035635 us; error 41 us; skew 500 ppm
I20260807 18:13:31.036018 31995 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:13:31 dist-test-slave-gkw7 krb5kdc[30876](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126411, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.154.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:13:31.039799 31995 init.cc:377] Logged in from keytab as oryx/127.18.154.2@KRBTEST.COM (short username oryx)
I20260807 18:13:31.041676 31995 webserver.cc:460] Webserver started at http://127.18.154.2:42043/ using document root <none> and password file <none>
I20260807 18:13:31.042434 31995 fs_manager.cc:359] Metadata directory not provided
I20260807 18:13:31.042600 31995 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:13:31.049094 31995 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 18:13:31.052053 32018 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:13:31.053571 31995 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 18:13:31.053758 31995 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/minicluster-data/ts-1/wal
uuid: "1f58e5c57f8d462dbb14b49d50af9c35"
format_stamp: "Formatted at 2026-08-07 18:13:28 on dist-test-slave-gkw7"
server_key: "99c487560e3a3c164d230e694de3bd7f"
server_key_iv: "0ae4cd0df78c383c3b6b6bc464a45a7e"
server_key_version: "encryptionkey@0"
I20260807 18:13:31.054911 31995 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:13:31.076330 31995 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:13:31.081696 31995 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:13:31.082001 31995 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:13:31.083173 31995 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:13:31.084757 31995 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:13:31.084883 31995 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:13:31.085022 31995 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:13:31.085101 31995 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:13:31.135727 31995 rpc_server.cc:307] RPC server started. Bound to: 127.18.154.2:36957
I20260807 18:13:31.135891 32173 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.154.2:36957 every 8 connection(s)
I20260807 18:13:31.137778 31995 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/minicluster-data/ts-1/data/info.pb
I20260807 18:13:31.140405 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 31995
I20260807 18:13:31.141458 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.154.3:34321
--local_ip_for_outbound_sockets=127.18.154.3
--tserver_master_addrs=127.18.154.62:36685
--webserver_port=36917
--webserver_interface=127.18.154.3
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.3.keytab
--principal=kudu/127.18.154.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.154.20:34183
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.18.154.3
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/krb5kdc/oryx_127.18.154.3.keytab with env {KRB5CCNAME=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 18:13:31 dist-test-slave-gkw7 krb5kdc[30876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.154.2@KRBTEST.COM for oryx/127.18.154.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:13:31 dist-test-slave-gkw7 krb5kdc[30876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.154.2@KRBTEST.COM for oryx/127.18.154.62@KRBTEST.COM, Server not found in Kerberos database
W20260807 18:13:31.520649 32181 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:13:31.521004 32181 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:13:31.521220 32181 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:13:31.529624 32181 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:13:31.529796 32181 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:13:31.530014 32181 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.154.3
I20260807 18:13:31.540715 32181 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.154.20:34183
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/krb5kdc/oryx_127.18.154.3.keytab
--principal=oryx/127.18.154.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.154.3:34321
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.154.3
--webserver_port=36917
--tserver_master_addrs=127.18.154.62:36685
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.154.3
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:13:31.542483 32181 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:13:31.543898 32181 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:13:31.563174 32191 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:13:31.570634 32190 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:13:31.579394 32195 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:13:31.582222 32181 server_base.cc:1034] running on GCE node
I20260807 18:13:31.583470 32181 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:13:31.585851 32181 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:13:31.587411 32181 hybrid_clock.cc:648] HybridClock initialized: now 1786126411587301 us; error 81 us; skew 500 ppm
I20260807 18:13:31.588012 32181 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:13:31 dist-test-slave-gkw7 krb5kdc[30876](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126411, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.154.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:13:31.594542 32181 init.cc:377] Logged in from keytab as oryx/127.18.154.3@KRBTEST.COM (short username oryx)
I20260807 18:13:31.597636 32181 webserver.cc:460] Webserver started at http://127.18.154.3:36917/ using document root <none> and password file <none>
I20260807 18:13:31.599054 32181 fs_manager.cc:359] Metadata directory not provided
I20260807 18:13:31.599280 32181 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:13:31.610782 32181 fs_manager.cc:709] Time spent opening directory manager: real 0.007s user 0.007s sys 0.000s
I20260807 18:13:31.616155 32204 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:13:31.618868 32181 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.006s sys 0.000s
I20260807 18:13:31.619346 32181 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/minicluster-data/ts-2/wal
uuid: "c2f6d5d122154c208c979ecaeab97274"
format_stamp: "Formatted at 2026-08-07 18:13:28 on dist-test-slave-gkw7"
server_key: "9fec385ce423619b5818cab37e153a16"
server_key_iv: "afabb28a8b5005ca10ce61356be8c64b"
server_key_version: "encryptionkey@0"
I20260807 18:13:31.621444 32181 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:13:31.723563 32181 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:13:31.732625 32181 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:13:31.733107 32181 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:13:31.735011 32181 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:13:31.737685 32181 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:13:31.737865 32181 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:13:31.738024 32181 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:13:31.738137 32181 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:13:31.805593 32181 rpc_server.cc:307] RPC server started. Bound to: 127.18.154.3:34321
I20260807 18:13:31.805891 32362 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.154.3:34321 every 8 connection(s)
I20260807 18:13:31.808538 32181 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786126328808105-19048-0/minicluster-data/ts-2/data/info.pb
I20260807 18:13:31.815488 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 32181
Aug 07 18:13:31 dist-test-slave-gkw7 krb5kdc[30876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.154.3@KRBTEST.COM for oryx/127.18.154.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:13:31 dist-test-slave-gkw7 krb5kdc[30876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.154.3@KRBTEST.COM for oryx/127.18.154.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:13:31.863867 32368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:31.822131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38918 (local address 127.18.154.62:36685)
0807 18:13:31.833554 (+ 11423us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:31.833565 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:31.833595 (+ 30us) server_negotiation.cc:407] Connection header received
0807 18:13:31.833690 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:31.833699 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:31.833805 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:31.833986 (+ 181us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:31.837415 (+ 3429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:31.839392 (+ 1977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:31.842480 (+ 3088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:31.843456 (+ 976us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:31.847145 (+ 3689us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:31.848979 (+ 1834us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:13:31.848997 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:13:31.849034 (+ 37us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:13:31.849118 (+ 84us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:13:31.855199 (+ 6081us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:31.856726 (+ 1527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:31.856743 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:31.856759 (+ 16us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:31.857097 (+ 338us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:31.857763 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:31.857771 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:31.857777 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:31.858768 (+ 991us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:13:31.861008 (+ 2240us) server_negotiation.cc:1035] Waiting for connection context
0807 18:13:31.861433 (+ 425us) server_negotiation.cc:299] Negotiation successful
0807 18:13:31.862239 (+ 806us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11161,"thread_start_us":12143,"threads_started":1}
W20260807 18:13:32.144820 31987 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.493s user 0.006s sys 0.009s
I20260807 18:13:32.147832 31987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:30.640720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:36685 (local address 127.18.154.1:39233)
0807 18:13:30.641617 (+ 897us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:30.641681 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0807 18:13:30.643686 (+ 2005us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:13:30.645180 (+ 1494us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:13:30.645190 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:13:30.645892 (+ 702us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:13:30.646469 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:30.646487 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:30.648127 (+ 1640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:30.648132 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:30.649466 (+ 1334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:30.649485 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:30.650703 (+ 1218us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:30.650713 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:30.650998 (+ 285us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:30.651768 (+ 770us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:13:30.651792 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:13:32.146157 (+1494365us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.154.62:36685: Server oryx/127.18.154.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":556,"thread_start_us":358,"threads_started":1}
W20260807 18:13:32.149211 31994 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:13:30.640715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.1:39233 (local address 127.18.154.62:36685)
0807 18:13:30.642699 (+ 1984us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:30.642719 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:30.642780 (+ 61us) server_negotiation.cc:407] Connection header received
0807 18:13:30.644283 (+ 1503us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:30.644297 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:30.644643 (+ 346us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:30.645020 (+ 377us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:30.646688 (+ 1668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:30.647930 (+ 1242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:30.649686 (+ 1756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:30.650437 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:30.650591 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:32.148729 (+1498138us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.154.1:39233: BlockingRecv error: recv got EOF from 127.18.154.1:39233 (error 108)
Metrics: {"server-negotiator.queue_time_us":1568,"thread_start_us":1311,"threads_started":1}
W20260807 18:13:32.151729 31985 heartbeater.cc:644] Failed to heartbeat to 127.18.154.62:36685 (0 consecutive failures): Not authorized: Failed to ping master at 127.18.154.62:36685: Client connection negotiation failed: client connection to 127.18.154.62:36685: Server oryx/127.18.154.62@KRBTEST.COM not found in Kerberos database
Aug 07 18:13:32 dist-test-slave-gkw7 krb5kdc[30876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.154.1@KRBTEST.COM for oryx/127.18.154.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:13:32 dist-test-slave-gkw7 krb5kdc[30876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.154.1@KRBTEST.COM for oryx/127.18.154.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:13:32.172065 31987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:32.152879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:36685 (local address 127.18.154.1:39861)
0807 18:13:32.153360 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:32.153391 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0807 18:13:32.153582 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:13:32.154461 (+ 879us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:13:32.154470 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:13:32.155781 (+ 1311us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:13:32.156120 (+ 339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:32.156151 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:32.158546 (+ 2395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:32.158556 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:32.161169 (+ 2613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:32.161207 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:32.163075 (+ 1868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:32.163085 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:32.163397 (+ 312us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:32.164497 (+ 1100us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:13:32.164533 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:13:32.171578 (+ 7045us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.154.62:36685: Server oryx/127.18.154.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":250}
W20260807 18:13:32.172406 31994 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:13:32.153430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.1:39861 (local address 127.18.154.62:36685)
0807 18:13:32.153717 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:32.153727 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:32.153757 (+ 30us) server_negotiation.cc:407] Connection header received
0807 18:13:32.153842 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:32.153851 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:32.153968 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:32.154133 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:32.156407 (+ 2274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:32.158315 (+ 1908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:32.161453 (+ 3138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:32.162505 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:32.162651 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:32.172012 (+ 9361us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.154.1:39861: BlockingRecv error: recv got EOF from 127.18.154.1:39861 (error 108)
Metrics: {"server-negotiator.queue_time_us":119}
Aug 07 18:13:32 dist-test-slave-gkw7 krb5kdc[30876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.154.1@KRBTEST.COM for oryx/127.18.154.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:13:32 dist-test-slave-gkw7 krb5kdc[30876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.154.1@KRBTEST.COM for oryx/127.18.154.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:13:32.196826 31987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:32.174295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:36685 (local address 127.18.154.1:36201)
0807 18:13:32.177918 (+ 3623us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:32.177951 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0807 18:13:32.178217 (+ 266us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:13:32.179036 (+ 819us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:13:32.179045 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:13:32.179878 (+ 833us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:13:32.180306 (+ 428us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:32.180326 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:32.182705 (+ 2379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:32.182714 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:32.185878 (+ 3164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:32.185928 (+ 50us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:32.187974 (+ 2046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:32.187984 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:32.188356 (+ 372us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:32.189615 (+ 1259us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:13:32.189649 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:13:32.196326 (+ 6677us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.154.62:36685: Server oryx/127.18.154.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":3453,"thread_start_us":11118,"threads_started":1}
W20260807 18:13:32.197134 31994 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:13:32.174837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.1:36201 (local address 127.18.154.62:36685)
0807 18:13:32.175088 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:32.175098 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:32.178057 (+ 2959us) server_negotiation.cc:407] Connection header received
0807 18:13:32.178417 (+ 360us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:32.178426 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:32.178541 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:32.178672 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:32.180557 (+ 1885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:32.182408 (+ 1851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:32.186285 (+ 3877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:32.187385 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:32.187532 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:32.196690 (+ 9158us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.154.1:36201: BlockingRecv error: recv got EOF from 127.18.154.1:36201 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
W20260807 18:13:32.197932 31985 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 18:13:32.547205 32178 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.370s user 0.011s sys 0.008s
W20260807 18:13:32.549039 32177 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:13:31.151156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.2:50095 (local address 127.18.154.62:36685)
0807 18:13:31.166791 (+ 15635us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:31.166800 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:31.166832 (+ 32us) server_negotiation.cc:407] Connection header received
0807 18:13:31.166909 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:31.166917 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:31.167027 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:31.168702 (+ 1675us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:31.169722 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:31.171569 (+ 1847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:31.174423 (+ 2854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:31.175433 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:31.175626 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:32.548531 (+1372905us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.154.2:50095: BlockingRecv error: recv got EOF from 127.18.154.2:50095 (error 108)
Metrics: {"server-negotiator.queue_time_us":15433,"thread_start_us":15984,"threads_started":1}
I20260807 18:13:32.549132 32178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:31.152032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:36685 (local address 127.18.154.2:50095)
0807 18:13:31.161607 (+ 9575us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:31.161704 (+ 97us) client_negotiation.cc:174] Beginning negotiation
0807 18:13:31.165614 (+ 3910us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:13:31.167589 (+ 1975us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:13:31.167604 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:13:31.168485 (+ 881us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:13:31.169436 (+ 951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:31.169465 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:31.171805 (+ 2340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:31.171814 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:31.174215 (+ 2401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:31.174234 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:31.175764 (+ 1530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:31.175775 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:31.176145 (+ 370us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:31.177225 (+ 1080us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:13:31.177271 (+ 46us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:13:32.548063 (+1370792us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.154.62:36685: Server oryx/127.18.154.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":8980,"thread_start_us":14096,"threads_started":1}
W20260807 18:13:32.550480 32174 heartbeater.cc:644] Failed to heartbeat to 127.18.154.62:36685 (0 consecutive failures): Not authorized: Failed to ping master at 127.18.154.62:36685: Client connection negotiation failed: client connection to 127.18.154.62:36685: Server oryx/127.18.154.62@KRBTEST.COM not found in Kerberos database
Aug 07 18:13:32 dist-test-slave-gkw7 krb5kdc[30876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.154.2@KRBTEST.COM for oryx/127.18.154.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:13:32 dist-test-slave-gkw7 krb5kdc[30876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.154.2@KRBTEST.COM for oryx/127.18.154.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:13:32.563593 32178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:32.551423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:36685 (local address 127.18.154.2:39683)
0807 18:13:32.551646 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:32.551667 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 18:13:32.551792 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:13:32.552607 (+ 815us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:13:32.552614 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:13:32.553097 (+ 483us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:13:32.553405 (+ 308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:32.553418 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:32.554943 (+ 1525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:32.554950 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:32.556357 (+ 1407us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:32.556372 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:32.557224 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:32.557231 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:32.557443 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:32.558340 (+ 897us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:13:32.558415 (+ 75us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:13:32.563269 (+ 4854us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.154.62:36685: Server oryx/127.18.154.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":79}
W20260807 18:13:32.563733 32177 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:13:32.551529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.2:39683 (local address 127.18.154.62:36685)
0807 18:13:32.552155 (+ 626us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:32.552164 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:32.552191 (+ 27us) server_negotiation.cc:407] Connection header received
0807 18:13:32.552273 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:32.552281 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:32.552393 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:32.552522 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:32.553615 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:32.554756 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:32.556541 (+ 1785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:32.556997 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:32.557183 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:32.563484 (+ 6301us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.154.2:39683: BlockingRecv error: recv got EOF from 127.18.154.2:39683 (error 108)
Metrics: {"server-negotiator.queue_time_us":422}
Aug 07 18:13:32 dist-test-slave-gkw7 krb5kdc[30876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.154.2@KRBTEST.COM for oryx/127.18.154.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:13:32 dist-test-slave-gkw7 krb5kdc[30876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.154.2@KRBTEST.COM for oryx/127.18.154.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:13:32.575198 32178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:32.564831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:36685 (local address 127.18.154.2:59401)
0807 18:13:32.565575 (+ 744us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:32.565601 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0807 18:13:32.565826 (+ 225us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:13:32.566243 (+ 417us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:13:32.566248 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:13:32.566659 (+ 411us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:13:32.566878 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:32.566887 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:32.568021 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:32.568026 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:32.569016 (+ 990us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:32.569027 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:32.569803 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:32.569809 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:32.570008 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:32.570638 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:13:32.570655 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:13:32.574879 (+ 4224us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.154.62:36685: Server oryx/127.18.154.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":532}
W20260807 18:13:32.575407 32177 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:13:32.565143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.2:59401 (local address 127.18.154.62:36685)
0807 18:13:32.565287 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:32.565291 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:32.565694 (+ 403us) server_negotiation.cc:407] Connection header received
0807 18:13:32.565982 (+ 288us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:32.565991 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:32.566079 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:32.566187 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:32.567013 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:32.567865 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:32.569176 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:32.569649 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:32.569774 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:32.575134 (+ 5360us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.154.2:59401: BlockingRecv error: recv got EOF from 127.18.154.2:59401 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
W20260807 18:13:32.575927 32174 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 18:13:32.992461 32365 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.148s user 0.007s sys 0.013s
W20260807 18:13:32.994048 32367 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:13:31.817683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.3:40353 (local address 127.18.154.62:36685)
0807 18:13:31.822259 (+ 4576us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:31.822269 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:31.827904 (+ 5635us) server_negotiation.cc:407] Connection header received
0807 18:13:31.828020 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:31.828029 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:31.828166 (+ 137us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:31.828311 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:31.832322 (+ 4011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:31.834324 (+ 2002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:31.841204 (+ 6880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:31.842198 (+ 994us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:31.842368 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:32.993735 (+1151367us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.154.3:40353: BlockingRecv error: recv got EOF from 127.18.154.3:40353 (error 108)
Metrics: {"server-negotiator.queue_time_us":4384,"thread_start_us":4163,"threads_started":1}
I20260807 18:13:32.994868 32365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:31.814261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:36685 (local address 127.18.154.3:40353)
0807 18:13:31.823215 (+ 8954us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:31.823293 (+ 78us) client_negotiation.cc:174] Beginning negotiation
0807 18:13:31.827265 (+ 3972us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:13:31.829257 (+ 1992us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:13:31.829277 (+ 20us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:13:31.830665 (+ 1388us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:13:31.831959 (+ 1294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:31.831996 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:31.837602 (+ 5606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:31.837611 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:31.840941 (+ 3330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:31.840965 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:31.842990 (+ 2025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:31.843000 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:31.843460 (+ 460us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:31.844685 (+ 1225us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:13:31.844743 (+ 58us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:13:32.993474 (+1148731us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.154.62:36685: Server oryx/127.18.154.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":8413,"thread_start_us":8610,"threads_started":1}
W20260807 18:13:32.995916 32363 heartbeater.cc:644] Failed to heartbeat to 127.18.154.62:36685 (0 consecutive failures): Not authorized: Failed to ping master at 127.18.154.62:36685: Client connection negotiation failed: client connection to 127.18.154.62:36685: Server oryx/127.18.154.62@KRBTEST.COM not found in Kerberos database
Aug 07 18:13:32 dist-test-slave-gkw7 krb5kdc[30876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.154.3@KRBTEST.COM for oryx/127.18.154.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:13:32 dist-test-slave-gkw7 krb5kdc[30876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.154.3@KRBTEST.COM for oryx/127.18.154.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:13:33.006978 32365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:32.996827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:36685 (local address 127.18.154.3:33865)
0807 18:13:32.997053 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:32.997089 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0807 18:13:32.997228 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:13:32.997621 (+ 393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:13:32.997626 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:13:32.998061 (+ 435us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:13:32.998290 (+ 229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:32.998299 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:32.999508 (+ 1209us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:32.999515 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:33.000648 (+ 1133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:33.000661 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:33.001476 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:33.001483 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:33.001714 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:33.002441 (+ 727us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:13:33.002462 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:13:33.006625 (+ 4163us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.154.62:36685: Server oryx/127.18.154.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":73}
W20260807 18:13:33.007115 32367 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:13:32.996856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.3:33865 (local address 127.18.154.62:36685)
0807 18:13:32.997135 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:32.997142 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:32.997162 (+ 20us) server_negotiation.cc:407] Connection header received
0807 18:13:32.997365 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:32.997371 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:32.997451 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:32.997611 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:32.998444 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:32.999357 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:33.000810 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:33.001281 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:33.001443 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:33.006876 (+ 5433us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.154.3:33865: BlockingRecv error: recv got EOF from 127.18.154.3:33865 (error 108)
Metrics: {"server-negotiator.queue_time_us":126}
Aug 07 18:13:33 dist-test-slave-gkw7 krb5kdc[30876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.154.3@KRBTEST.COM for oryx/127.18.154.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:13:33 dist-test-slave-gkw7 krb5kdc[30876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.154.3@KRBTEST.COM for oryx/127.18.154.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:13:33.016611 32365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:33.007894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:36685 (local address 127.18.154.3:56117)
0807 18:13:33.008039 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:33.008066 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0807 18:13:33.008200 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:13:33.008557 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:13:33.008562 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:13:33.008906 (+ 344us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:13:33.009068 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:33.009078 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:33.010220 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:33.010226 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:33.011211 (+ 985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:33.011223 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:33.012000 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:33.012005 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:33.012193 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:33.012855 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:13:33.012874 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:13:33.016401 (+ 3527us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.154.62:36685: Server oryx/127.18.154.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":37}
W20260807 18:13:33.016898 32367 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:13:33.007960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.3:56117 (local address 127.18.154.62:36685)
0807 18:13:33.008245 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:33.008253 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:33.008281 (+ 28us) server_negotiation.cc:407] Connection header received
0807 18:13:33.008333 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:33.008337 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:33.008408 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:33.008516 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:33.009214 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:33.010071 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:33.011372 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:33.011843 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:33.011987 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:33.016631 (+ 4644us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.154.3:56117: BlockingRecv error: recv got EOF from 127.18.154.3:56117 (error 108)
Metrics: {"server-negotiator.queue_time_us":172}
W20260807 18:13:33.016991 32363 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 07 18:13:33 dist-test-slave-gkw7 krb5kdc[30876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.154.1@KRBTEST.COM for oryx/127.18.154.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:13:33 dist-test-slave-gkw7 krb5kdc[30876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.154.1@KRBTEST.COM for oryx/127.18.154.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:13:33.208268 32373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:33.198693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:36685 (local address 127.18.154.1:37221)
0807 18:13:33.199235 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:33.199259 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 18:13:33.199378 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:13:33.199711 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:13:33.199715 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:13:33.200170 (+ 455us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:13:33.200324 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:33.200332 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:33.201327 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:33.201331 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:33.202379 (+ 1048us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:33.202391 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:33.203057 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:33.203062 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:33.203194 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:33.203871 (+ 677us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:13:33.203888 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:13:33.207921 (+ 4033us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.154.62:36685: Server oryx/127.18.154.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":418,"thread_start_us":304,"threads_started":1}
W20260807 18:13:33.208418 32367 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:13:33.198795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.1:37221 (local address 127.18.154.62:36685)
0807 18:13:33.199003 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:33.199012 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:33.199340 (+ 328us) server_negotiation.cc:407] Connection header received
0807 18:13:33.199485 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:33.199488 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:33.199572 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:33.199663 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:33.200458 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:33.201179 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:33.202506 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:33.202900 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:33.202993 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:33.208157 (+ 5164us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.154.1:37221: BlockingRecv error: recv got EOF from 127.18.154.1:37221 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Aug 07 18:13:33 dist-test-slave-gkw7 krb5kdc[30876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.154.2@KRBTEST.COM for oryx/127.18.154.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:13:33 dist-test-slave-gkw7 krb5kdc[30876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.154.2@KRBTEST.COM for oryx/127.18.154.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:13:33.591076 32374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:33.576547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:36685 (local address 127.18.154.2:54773)
0807 18:13:33.577119 (+ 572us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:33.577134 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 18:13:33.577327 (+ 193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:13:33.577708 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:13:33.577713 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:13:33.578200 (+ 487us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:13:33.578367 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:33.578376 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:33.579510 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:33.579520 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:33.580561 (+ 1041us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:33.580574 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:33.581329 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:33.581335 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:33.581510 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:33.587071 (+ 5561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:13:33.587091 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:13:33.590718 (+ 3627us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.154.62:36685: Server oryx/127.18.154.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":479,"thread_start_us":332,"threads_started":1}
W20260807 18:13:33.591244 32367 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:13:33.576641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.2:54773 (local address 127.18.154.62:36685)
0807 18:13:33.576888 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:33.576897 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:33.577230 (+ 333us) server_negotiation.cc:407] Connection header received
0807 18:13:33.577477 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:33.577481 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:33.577560 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:33.577672 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:33.578515 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:33.579360 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:33.580700 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:33.581135 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:33.581292 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:33.590977 (+ 9685us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.154.2:54773: BlockingRecv error: recv got EOF from 127.18.154.2:54773 (error 108)
Metrics: {"server-negotiator.queue_time_us":88}
Aug 07 18:13:34 dist-test-slave-gkw7 krb5kdc[30876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.154.3@KRBTEST.COM for oryx/127.18.154.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:13:34 dist-test-slave-gkw7 krb5kdc[30876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.154.3@KRBTEST.COM for oryx/127.18.154.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:13:34.028228 32376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:34.017807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:36685 (local address 127.18.154.3:42693)
0807 18:13:34.018504 (+ 697us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:34.018523 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 18:13:34.018653 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:13:34.018982 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:13:34.018986 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:13:34.019502 (+ 516us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:13:34.019666 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:34.019675 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:34.020785 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:34.020789 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:34.022013 (+ 1224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:34.022033 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:34.022942 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:34.022950 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:34.023204 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:34.023935 (+ 731us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:13:34.023959 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:13:34.027991 (+ 4032us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.154.62:36685: Server oryx/127.18.154.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":558,"thread_start_us":409,"threads_started":1}
W20260807 18:13:34.028508 32367 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:13:34.017848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.3:42693 (local address 127.18.154.62:36685)
0807 18:13:34.018022 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:34.018027 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:34.018603 (+ 576us) server_negotiation.cc:407] Connection header received
0807 18:13:34.018759 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:34.018763 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:34.018842 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:34.018930 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:34.019793 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:34.020596 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:34.022188 (+ 1592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:34.022707 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:34.022852 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:34.028249 (+ 5397us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.154.3:42693: BlockingRecv error: recv got EOF from 127.18.154.3:42693 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 07 18:13:34 dist-test-slave-gkw7 krb5kdc[30876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.154.1@KRBTEST.COM for oryx/127.18.154.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:13:34 dist-test-slave-gkw7 krb5kdc[30876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.154.1@KRBTEST.COM for oryx/127.18.154.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:13:34.220206 32379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:34.209408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:36685 (local address 127.18.154.1:53723)
0807 18:13:34.210295 (+ 887us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:34.210316 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 18:13:34.210586 (+ 270us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:13:34.211020 (+ 434us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:13:34.211025 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:13:34.211583 (+ 558us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:13:34.211827 (+ 244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:34.211841 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:34.213458 (+ 1617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:34.213466 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:34.214703 (+ 1237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:34.214717 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:34.215444 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:34.215448 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:34.215589 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:34.216184 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:13:34.216203 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:13:34.219909 (+ 3706us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.154.62:36685: Server oryx/127.18.154.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":705,"thread_start_us":594,"threads_started":1}
W20260807 18:13:34.220633 32367 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:13:34.209454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.1:53723 (local address 127.18.154.62:36685)
0807 18:13:34.209727 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:34.209733 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:34.210483 (+ 750us) server_negotiation.cc:407] Connection header received
0807 18:13:34.210733 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:34.210739 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:34.210855 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:34.210956 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:34.212013 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:34.213292 (+ 1279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:34.214845 (+ 1553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:34.215296 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:34.215406 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:34.220286 (+ 4880us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.154.1:53723: BlockingRecv error: recv got EOF from 127.18.154.1:53723 (error 108)
Metrics: {"server-negotiator.queue_time_us":106}
Aug 07 18:13:34 dist-test-slave-gkw7 krb5kdc[30876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.154.2@KRBTEST.COM for oryx/127.18.154.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:13:34 dist-test-slave-gkw7 krb5kdc[30876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.154.2@KRBTEST.COM for oryx/127.18.154.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:13:34.602423 32380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:34.592246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:36685 (local address 127.18.154.2:56795)
0807 18:13:34.592937 (+ 691us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:34.592955 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 18:13:34.593110 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:13:34.593513 (+ 403us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:13:34.593517 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:13:34.593991 (+ 474us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:13:34.594198 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:34.594207 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:34.595260 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:34.595264 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:34.596312 (+ 1048us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:34.596325 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:34.597113 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:34.597119 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:34.597272 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:34.598016 (+ 744us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:13:34.598037 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:13:34.602135 (+ 4098us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.154.62:36685: Server oryx/127.18.154.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":558,"thread_start_us":433,"threads_started":1}
W20260807 18:13:34.602596 32367 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:13:34.592226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.2:56795 (local address 127.18.154.62:36685)
0807 18:13:34.592447 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:34.592452 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:34.593078 (+ 626us) server_negotiation.cc:407] Connection header received
0807 18:13:34.593231 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:34.593235 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:34.593347 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:34.593444 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:34.594342 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:34.595099 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:34.596484 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:34.596937 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:34.597085 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:34.602379 (+ 5294us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.154.2:56795: BlockingRecv error: recv got EOF from 127.18.154.2:56795 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
Aug 07 18:13:35 dist-test-slave-gkw7 krb5kdc[30876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.154.3@KRBTEST.COM for oryx/127.18.154.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:13:35 dist-test-slave-gkw7 krb5kdc[30876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.154.3@KRBTEST.COM for oryx/127.18.154.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:13:35.038986 32381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:35.029355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:36685 (local address 127.18.154.3:42203)
0807 18:13:35.029983 (+ 628us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:35.030001 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 18:13:35.030118 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:13:35.030532 (+ 414us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:13:35.030535 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:13:35.031114 (+ 579us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:13:35.031314 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:35.031324 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:35.032429 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:35.032432 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:35.033373 (+ 941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:35.033384 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:35.034186 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:35.034195 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:35.034334 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:35.035123 (+ 789us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:13:35.035150 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:13:35.038741 (+ 3591us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.154.62:36685: Server oryx/127.18.154.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":510,"thread_start_us":394,"threads_started":1}
W20260807 18:13:35.039216 32367 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:13:35.029606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.3:42203 (local address 127.18.154.62:36685)
0807 18:13:35.029802 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:35.029808 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:35.030112 (+ 304us) server_negotiation.cc:407] Connection header received
0807 18:13:35.030245 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:35.030253 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:35.030372 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:35.030473 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:35.031456 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:35.032286 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:35.033496 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:35.033980 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:35.034087 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:35.038934 (+ 4847us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.154.3:42203: BlockingRecv error: recv got EOF from 127.18.154.3:42203 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 07 18:13:35 dist-test-slave-gkw7 krb5kdc[30876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.154.1@KRBTEST.COM for oryx/127.18.154.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:13:35 dist-test-slave-gkw7 krb5kdc[30876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.154.1@KRBTEST.COM for oryx/127.18.154.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:13:35.230743 32382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:35.221151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:36685 (local address 127.18.154.1:51475)
0807 18:13:35.221763 (+ 612us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:35.221796 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0807 18:13:35.221971 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:13:35.222343 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:13:35.222347 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:13:35.222772 (+ 425us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:13:35.222950 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:35.222959 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:35.224041 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:35.224047 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:35.224994 (+ 947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:35.225006 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:35.225802 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:35.225808 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:35.225992 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:35.226583 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:13:35.226600 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:13:35.230452 (+ 3852us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.154.62:36685: Server oryx/127.18.154.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":504,"thread_start_us":291,"threads_started":1}
W20260807 18:13:35.231096 32367 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:13:35.221212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.1:51475 (local address 127.18.154.62:36685)
0807 18:13:35.221425 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:35.221435 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:35.221891 (+ 456us) server_negotiation.cc:407] Connection header received
0807 18:13:35.222092 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:35.222097 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:35.222200 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:35.222310 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:35.223098 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:35.223900 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:35.225146 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:35.225655 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:35.225788 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:35.230839 (+ 5051us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.154.1:51475: BlockingRecv error: recv got EOF from 127.18.154.1:51475 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 07 18:13:35 dist-test-slave-gkw7 krb5kdc[30876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.154.2@KRBTEST.COM for oryx/127.18.154.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:13:35 dist-test-slave-gkw7 krb5kdc[30876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.154.2@KRBTEST.COM for oryx/127.18.154.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:13:35.612300 32383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:35.603351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:36685 (local address 127.18.154.2:51481)
0807 18:13:35.603888 (+ 537us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:35.603904 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 18:13:35.604021 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:13:35.604334 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:13:35.604337 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:13:35.604701 (+ 364us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:13:35.604897 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:35.604907 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:35.605985 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:35.605989 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:35.606943 (+ 954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:35.606955 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:35.607625 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:35.607628 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:35.607758 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:35.608421 (+ 663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:13:35.608440 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:13:35.612046 (+ 3606us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.154.62:36685: Server oryx/127.18.154.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":424,"thread_start_us":305,"threads_started":1}
W20260807 18:13:35.612581 32367 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:13:35.603453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.2:51481 (local address 127.18.154.62:36685)
0807 18:13:35.603695 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:35.603703 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:35.603987 (+ 284us) server_negotiation.cc:407] Connection header received
0807 18:13:35.604126 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:35.604129 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:35.604196 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:35.604285 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:35.605031 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:35.605845 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:35.607065 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:35.607496 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:35.607594 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:35.612279 (+ 4685us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.154.2:51481: BlockingRecv error: recv got EOF from 127.18.154.2:51481 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
Aug 07 18:13:36 dist-test-slave-gkw7 krb5kdc[30876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.154.3@KRBTEST.COM for oryx/127.18.154.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:13:36 dist-test-slave-gkw7 krb5kdc[30876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.154.3@KRBTEST.COM for oryx/127.18.154.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:13:36.050069 32384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:36.039931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:36685 (local address 127.18.154.3:47503)
0807 18:13:36.040541 (+ 610us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:36.040558 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 18:13:36.040704 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:13:36.041039 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:13:36.041043 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:13:36.041411 (+ 368us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:13:36.041621 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:36.041630 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:36.042672 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:36.042676 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:36.043626 (+ 950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:36.043638 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:36.044324 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:36.044328 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:36.044466 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:36.045135 (+ 669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:13:36.045158 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:13:36.049600 (+ 4442us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.154.62:36685: Server oryx/127.18.154.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":489,"thread_start_us":371,"threads_started":1}
W20260807 18:13:36.050143 32367 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:13:36.040039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.3:47503 (local address 127.18.154.62:36685)
0807 18:13:36.040230 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:36.040234 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:36.040646 (+ 412us) server_negotiation.cc:407] Connection header received
0807 18:13:36.040803 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:36.040807 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:36.040871 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:36.040959 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:36.041800 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:36.042527 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:36.043748 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:36.044153 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:36.044254 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:36.049862 (+ 5608us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.154.3:47503: BlockingRecv error: recv got EOF from 127.18.154.3:47503 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 07 18:13:36 dist-test-slave-gkw7 krb5kdc[30876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.154.1@KRBTEST.COM for oryx/127.18.154.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:13:36 dist-test-slave-gkw7 krb5kdc[30876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.154.1@KRBTEST.COM for oryx/127.18.154.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:13:36.242297 32385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:36.231956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:36685 (local address 127.18.154.1:44567)
0807 18:13:36.232498 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:36.232521 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 18:13:36.232641 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:13:36.232963 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:13:36.232966 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:13:36.233400 (+ 434us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:13:36.233637 (+ 237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:36.233648 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:36.235169 (+ 1521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:36.235177 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:36.236280 (+ 1103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:36.236293 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:36.237063 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:36.237075 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:36.237291 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:36.237967 (+ 676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:13:36.237988 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:13:36.241869 (+ 3881us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.154.62:36685: Server oryx/127.18.154.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":424,"thread_start_us":312,"threads_started":1}
W20260807 18:13:36.242518 32367 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:13:36.232049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.1:44567 (local address 127.18.154.62:36685)
0807 18:13:36.232279 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:36.232285 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:36.232597 (+ 312us) server_negotiation.cc:407] Connection header received
0807 18:13:36.232747 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:36.232751 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:36.232813 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:36.232907 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:36.233840 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:36.234952 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:36.236461 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:36.236895 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:36.237058 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:36.242192 (+ 5134us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.154.1:44567: BlockingRecv error: recv got EOF from 127.18.154.1:44567 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
Aug 07 18:13:36 dist-test-slave-gkw7 krb5kdc[30876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.154.2@KRBTEST.COM for oryx/127.18.154.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:13:36 dist-test-slave-gkw7 krb5kdc[30876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.154.2@KRBTEST.COM for oryx/127.18.154.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:13:36.623615 32386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:36.613486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:36685 (local address 127.18.154.2:59739)
0807 18:13:36.614161 (+ 675us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:36.614181 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 18:13:36.614376 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:13:36.614808 (+ 432us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:13:36.614813 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:13:36.615272 (+ 459us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:13:36.615457 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:36.615467 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:36.616570 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:36.616579 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:36.617712 (+ 1133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:36.617724 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:36.618686 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:36.618692 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:36.618899 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:36.619629 (+ 730us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:13:36.619649 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:13:36.623360 (+ 3711us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.154.62:36685: Server oryx/127.18.154.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":571,"thread_start_us":362,"threads_started":1}
W20260807 18:13:36.623785 32367 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:13:36.613632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.2:59739 (local address 127.18.154.62:36685)
0807 18:13:36.613903 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:36.613912 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:36.614296 (+ 384us) server_negotiation.cc:407] Connection header received
0807 18:13:36.614542 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:36.614555 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:36.614627 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:36.614763 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:36.615631 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:36.616415 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:36.617883 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:36.618496 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:36.618661 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:36.623568 (+ 4907us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.154.2:59739: BlockingRecv error: recv got EOF from 127.18.154.2:59739 (error 108)
Metrics: {"server-negotiator.queue_time_us":96}
Aug 07 18:13:37 dist-test-slave-gkw7 krb5kdc[30876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.154.3@KRBTEST.COM for oryx/127.18.154.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:13:37 dist-test-slave-gkw7 krb5kdc[30876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.154.3@KRBTEST.COM for oryx/127.18.154.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:13:37.061225 32387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:37.051309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:36685 (local address 127.18.154.3:59585)
0807 18:13:37.051929 (+ 620us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:37.051947 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 18:13:37.052122 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:13:37.052563 (+ 441us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:13:37.052568 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:13:37.053160 (+ 592us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:13:37.053373 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:37.053388 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:37.054475 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:37.054481 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:37.055486 (+ 1005us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:37.055497 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:37.056204 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:37.056210 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:37.056391 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:37.057005 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:13:37.057022 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:13:37.060861 (+ 3839us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.154.62:36685: Server oryx/127.18.154.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":497,"thread_start_us":352,"threads_started":1}
W20260807 18:13:37.061365 32367 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:13:37.051385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.3:59585 (local address 127.18.154.62:36685)
0807 18:13:37.051659 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:37.051666 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:37.052068 (+ 402us) server_negotiation.cc:407] Connection header received
0807 18:13:37.052278 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:37.052283 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:37.052400 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:37.052528 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:37.053496 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:37.054339 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:37.055637 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:37.056045 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:37.056190 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:37.061079 (+ 4889us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.154.3:59585: BlockingRecv error: recv got EOF from 127.18.154.3:59585 (error 108)
Metrics: {"server-negotiator.queue_time_us":94}
Aug 07 18:13:37 dist-test-slave-gkw7 krb5kdc[30876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.154.1@KRBTEST.COM for oryx/127.18.154.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:13:37 dist-test-slave-gkw7 krb5kdc[30876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.154.1@KRBTEST.COM for oryx/127.18.154.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:13:37.252964 32388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:37.243329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:36685 (local address 127.18.154.1:41441)
0807 18:13:37.243913 (+ 584us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:37.243935 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 18:13:37.244107 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:13:37.244469 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:13:37.244474 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:13:37.244958 (+ 484us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:13:37.245118 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:37.245127 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:37.246311 (+ 1184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:37.246319 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:37.247279 (+ 960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:37.247301 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:37.247986 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:37.247992 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:37.248176 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:37.248905 (+ 729us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:13:37.248930 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:13:37.252611 (+ 3681us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.154.62:36685: Server oryx/127.18.154.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":475,"thread_start_us":318,"threads_started":1}
W20260807 18:13:37.253108 32367 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:13:37.243474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.1:41441 (local address 127.18.154.62:36685)
0807 18:13:37.243740 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:37.243746 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:37.244032 (+ 286us) server_negotiation.cc:407] Connection header received
0807 18:13:37.244247 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:37.244253 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:37.244327 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:37.244435 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:37.245296 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:37.246155 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:37.247435 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:37.247832 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:37.247970 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:37.252848 (+ 4878us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.154.1:41441: BlockingRecv error: recv got EOF from 127.18.154.1:41441 (error 108)
Metrics: {"server-negotiator.queue_time_us":98}
Aug 07 18:13:37 dist-test-slave-gkw7 krb5kdc[30876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.154.2@KRBTEST.COM for oryx/127.18.154.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:13:37 dist-test-slave-gkw7 krb5kdc[30876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.154.2@KRBTEST.COM for oryx/127.18.154.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:13:37.635145 32390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:37.624646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:36685 (local address 127.18.154.2:52127)
0807 18:13:37.625565 (+ 919us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:37.625583 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 18:13:37.625705 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:13:37.626044 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:13:37.626047 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:13:37.626638 (+ 591us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:13:37.626832 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:37.626843 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:37.627985 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:37.627990 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:37.629169 (+ 1179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:37.629183 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:37.630147 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:37.630152 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:37.630290 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:37.630954 (+ 664us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:13:37.630974 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:13:37.634859 (+ 3885us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.154.62:36685: Server oryx/127.18.154.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":762,"thread_start_us":631,"threads_started":1}
W20260807 18:13:37.635303 32367 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:13:37.624813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.2:52127 (local address 127.18.154.62:36685)
0807 18:13:37.625018 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:37.625023 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:37.625673 (+ 650us) server_negotiation.cc:407] Connection header received
0807 18:13:37.625826 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:37.625830 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:37.625903 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:37.625997 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:37.626999 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:37.627813 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:37.629344 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:37.629994 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:37.630103 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:37.635052 (+ 4949us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.154.2:52127: BlockingRecv error: recv got EOF from 127.18.154.2:52127 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Aug 07 18:13:38 dist-test-slave-gkw7 krb5kdc[30876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.154.3@KRBTEST.COM for oryx/127.18.154.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:13:38 dist-test-slave-gkw7 krb5kdc[30876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.154.3@KRBTEST.COM for oryx/127.18.154.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:13:38.072731 32391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:38.062295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:36685 (local address 127.18.154.3:55281)
0807 18:13:38.062903 (+ 608us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:38.062920 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 18:13:38.063100 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:13:38.063565 (+ 465us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:13:38.063572 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:13:38.064073 (+ 501us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:13:38.064230 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:38.064239 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:38.065347 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:38.065355 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:38.066452 (+ 1097us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:38.066465 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:38.067270 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:38.067277 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:38.067480 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:38.068238 (+ 758us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:13:38.068260 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:13:38.072387 (+ 4127us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.154.62:36685: Server oryx/127.18.154.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":496,"thread_start_us":343,"threads_started":1}
W20260807 18:13:38.072861 32367 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:13:38.062348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.3:55281 (local address 127.18.154.62:36685)
0807 18:13:38.062663 (+ 315us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:38.062673 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:38.063025 (+ 352us) server_negotiation.cc:407] Connection header received
0807 18:13:38.063265 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:38.063278 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:38.063396 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:38.063518 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:38.064392 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:38.065205 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:38.066618 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:38.067096 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:38.067250 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:38.072609 (+ 5359us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.154.3:55281: BlockingRecv error: recv got EOF from 127.18.154.3:55281 (error 108)
Metrics: {"server-negotiator.queue_time_us":111}
Aug 07 18:13:38 dist-test-slave-gkw7 krb5kdc[30876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.154.1@KRBTEST.COM for oryx/127.18.154.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:13:38 dist-test-slave-gkw7 krb5kdc[30876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.154.1@KRBTEST.COM for oryx/127.18.154.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:13:38.264323 32393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:38.254209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:36685 (local address 127.18.154.1:40523)
0807 18:13:38.254776 (+ 567us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:38.254797 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 18:13:38.254967 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:13:38.255352 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:13:38.255355 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:13:38.255745 (+ 390us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:13:38.255904 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:38.255913 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:38.257023 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:38.257031 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:38.258286 (+ 1255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:38.258301 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:38.259120 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:38.259125 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:38.259261 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:38.260101 (+ 840us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:13:38.260123 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:13:38.264049 (+ 3926us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.154.62:36685: Server oryx/127.18.154.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":438,"thread_start_us":306,"threads_started":1}
W20260807 18:13:38.264554 32367 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:13:38.254453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.1:40523 (local address 127.18.154.62:36685)
0807 18:13:38.254662 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:38.254667 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:38.254917 (+ 250us) server_negotiation.cc:407] Connection header received
0807 18:13:38.255125 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:38.255129 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:38.255204 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:38.255305 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:38.256063 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:38.256848 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:38.258460 (+ 1612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:38.258948 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:38.259066 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:38.264300 (+ 5234us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.154.1:40523: BlockingRecv error: recv got EOF from 127.18.154.1:40523 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Aug 07 18:13:38 dist-test-slave-gkw7 krb5kdc[30876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.154.2@KRBTEST.COM for oryx/127.18.154.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:13:38 dist-test-slave-gkw7 krb5kdc[30876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.154.2@KRBTEST.COM for oryx/127.18.154.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:13:38.649927 32394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:38.636244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:36685 (local address 127.18.154.2:34245)
0807 18:13:38.636812 (+ 568us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:38.636830 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 18:13:38.636951 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:13:38.637320 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:13:38.637323 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:13:38.637764 (+ 441us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:13:38.637934 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:38.637942 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:38.639352 (+ 1410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:38.639356 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:38.640330 (+ 974us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:38.640343 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:38.641030 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:38.641035 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:38.641190 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:38.645907 (+ 4717us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:13:38.645927 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:13:38.649585 (+ 3658us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.154.62:36685: Server oryx/127.18.154.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":437,"thread_start_us":313,"threads_started":1}
W20260807 18:13:38.650055 32367 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:13:38.636329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.2:34245 (local address 127.18.154.62:36685)
0807 18:13:38.636605 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:38.636612 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:38.636919 (+ 307us) server_negotiation.cc:407] Connection header received
0807 18:13:38.637074 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:38.637080 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:38.637165 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:38.637280 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:38.638078 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:38.639185 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:38.640461 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:38.640891 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:38.640986 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:38.649820 (+ 8834us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.154.2:34245: BlockingRecv error: recv got EOF from 127.18.154.2:34245 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Aug 07 18:13:39 dist-test-slave-gkw7 krb5kdc[30876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.154.3@KRBTEST.COM for oryx/127.18.154.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:13:39 dist-test-slave-gkw7 krb5kdc[30876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.154.3@KRBTEST.COM for oryx/127.18.154.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:13:39.083897 32395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:39.073883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:36685 (local address 127.18.154.3:44889)
0807 18:13:39.074486 (+ 603us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:39.074507 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 18:13:39.074658 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:13:39.075045 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:13:39.075049 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:13:39.075513 (+ 464us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:13:39.075728 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:39.075740 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:39.076880 (+ 1140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:39.076884 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:39.077934 (+ 1050us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:39.077947 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:39.078851 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:39.078856 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:39.078996 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:39.079624 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:13:39.079681 (+ 57us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:13:39.083612 (+ 3931us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.154.62:36685: Server oryx/127.18.154.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":472,"thread_start_us":345,"threads_started":1}
W20260807 18:13:39.084239 32367 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:13:39.073959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.3:44889 (local address 127.18.154.62:36685)
0807 18:13:39.074188 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:39.074193 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:39.074596 (+ 403us) server_negotiation.cc:407] Connection header received
0807 18:13:39.074772 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:39.074775 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:39.074841 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:39.074938 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:39.075910 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:39.076726 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:39.078096 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:39.078566 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:39.078801 (+ 235us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:39.083835 (+ 5034us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.154.3:44889: BlockingRecv error: recv got EOF from 127.18.154.3:44889 (error 108)
Metrics: {"server-negotiator.queue_time_us":87}
Aug 07 18:13:39 dist-test-slave-gkw7 krb5kdc[30876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.154.1@KRBTEST.COM for oryx/127.18.154.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:13:39 dist-test-slave-gkw7 krb5kdc[30876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.154.1@KRBTEST.COM for oryx/127.18.154.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:13:39.275544 32396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:39.265259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:36685 (local address 127.18.154.1:58047)
0807 18:13:39.265927 (+ 668us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:39.265949 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 18:13:39.266063 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:13:39.266372 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:13:39.266375 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:13:39.266818 (+ 443us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:13:39.267025 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:39.267033 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:39.268018 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:39.268021 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:39.269002 (+ 981us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:39.269013 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:39.269809 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:39.269815 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:39.269997 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:39.270772 (+ 775us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:13:39.270797 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:13:39.275137 (+ 4340us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.154.62:36685: Server oryx/127.18.154.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":543,"thread_start_us":315,"threads_started":1}
W20260807 18:13:39.275671 32367 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:13:39.265358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.1:58047 (local address 127.18.154.62:36685)
0807 18:13:39.265561 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:39.265570 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:39.266021 (+ 451us) server_negotiation.cc:407] Connection header received
0807 18:13:39.266157 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:39.266160 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:39.266242 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:39.266323 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:39.267148 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:39.267893 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:39.269123 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:39.269636 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:39.269782 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:39.275393 (+ 5611us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.154.1:58047: BlockingRecv error: recv got EOF from 127.18.154.1:58047 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 07 18:13:39 dist-test-slave-gkw7 krb5kdc[30876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.154.2@KRBTEST.COM for oryx/127.18.154.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:13:39 dist-test-slave-gkw7 krb5kdc[30876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.154.2@KRBTEST.COM for oryx/127.18.154.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:13:39.661497 32400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:39.650964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:36685 (local address 127.18.154.2:55709)
0807 18:13:39.651736 (+ 772us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:39.651754 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 18:13:39.651880 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:13:39.652237 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:13:39.652240 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:13:39.652738 (+ 498us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:13:39.652937 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:39.652947 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:39.654237 (+ 1290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:39.654242 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:39.655240 (+ 998us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:39.655252 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:39.656000 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:39.656004 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:39.656122 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:39.656839 (+ 717us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:13:39.656859 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:13:39.661237 (+ 4378us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.154.62:36685: Server oryx/127.18.154.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":639,"thread_start_us":380,"threads_started":1}
W20260807 18:13:39.661947 32367 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:13:39.651084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.2:55709 (local address 127.18.154.62:36685)
0807 18:13:39.651296 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:39.651301 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:39.651861 (+ 560us) server_negotiation.cc:407] Connection header received
0807 18:13:39.651994 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:39.651998 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:39.652103 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:39.652194 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:39.653140 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:39.654083 (+ 943us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:39.655392 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:39.655852 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:39.655961 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:39.661511 (+ 5550us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.154.2:55709: BlockingRecv error: recv got EOF from 127.18.154.2:55709 (error 108)
Metrics: {"server-negotiator.queue_time_us":87}
Aug 07 18:13:40 dist-test-slave-gkw7 krb5kdc[30876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.154.3@KRBTEST.COM for oryx/127.18.154.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:13:40 dist-test-slave-gkw7 krb5kdc[30876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.154.3@KRBTEST.COM for oryx/127.18.154.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:13:40.096040 32401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:40.085083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:36685 (local address 127.18.154.3:42509)
0807 18:13:40.085908 (+ 825us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:40.085932 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 18:13:40.086114 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:13:40.086494 (+ 380us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:13:40.086500 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:13:40.087126 (+ 626us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:13:40.087383 (+ 257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:40.087397 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:40.088542 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:40.088549 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:40.090159 (+ 1610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:40.090171 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:40.090911 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:40.090917 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:40.091080 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:40.091775 (+ 695us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:13:40.091792 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:13:40.095693 (+ 3901us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.154.62:36685: Server oryx/127.18.154.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":687,"thread_start_us":321,"threads_started":1}
W20260807 18:13:40.096114 32367 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:13:40.085186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.3:42509 (local address 127.18.154.62:36685)
0807 18:13:40.085409 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:40.085418 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:40.086049 (+ 631us) server_negotiation.cc:407] Connection header received
0807 18:13:40.086271 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:40.086278 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:40.086357 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:40.086439 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:40.087542 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:40.088409 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:40.090303 (+ 1894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:40.090753 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:40.090896 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:40.095894 (+ 4998us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.154.3:42509: BlockingRecv error: recv got EOF from 127.18.154.3:42509 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Aug 07 18:13:40 dist-test-slave-gkw7 krb5kdc[30876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.154.1@KRBTEST.COM for oryx/127.18.154.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:13:40 dist-test-slave-gkw7 krb5kdc[30876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.154.1@KRBTEST.COM for oryx/127.18.154.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:13:40.286743 32402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:40.276769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:36685 (local address 127.18.154.1:36487)
0807 18:13:40.277396 (+ 627us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:40.277418 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 18:13:40.277566 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:13:40.277995 (+ 429us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:13:40.277998 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:13:40.278434 (+ 436us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:13:40.278605 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:40.278613 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:40.279800 (+ 1187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:40.279804 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:40.280815 (+ 1011us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:40.280827 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:40.281507 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:40.281511 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:40.281688 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:40.282345 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:13:40.282362 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:13:40.286375 (+ 4013us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.154.62:36685: Server oryx/127.18.154.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":494,"thread_start_us":308,"threads_started":1}
W20260807 18:13:40.286882 32367 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:13:40.276992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.1:36487 (local address 127.18.154.62:36685)
0807 18:13:40.277214 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:40.277227 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:40.277509 (+ 282us) server_negotiation.cc:407] Connection header received
0807 18:13:40.277740 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:40.277745 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:40.277840 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:40.277957 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:40.278788 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:40.279686 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:40.280945 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:40.281355 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:40.281455 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:40.286599 (+ 5144us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.154.1:36487: BlockingRecv error: recv got EOF from 127.18.154.1:36487 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
Aug 07 18:13:40 dist-test-slave-gkw7 krb5kdc[30876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.154.2@KRBTEST.COM for oryx/127.18.154.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:13:40 dist-test-slave-gkw7 krb5kdc[30876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.154.2@KRBTEST.COM for oryx/127.18.154.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:13:40.672341 32403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:40.662724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:36685 (local address 127.18.154.2:42925)
0807 18:13:40.663286 (+ 562us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:40.663304 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 18:13:40.663427 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:13:40.663791 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:13:40.663794 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:13:40.664252 (+ 458us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:13:40.664430 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:40.664440 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:40.665674 (+ 1234us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:40.665679 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:40.666673 (+ 994us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:40.666685 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:40.667495 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:40.667500 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:40.667620 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:40.668228 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:13:40.668246 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:13:40.671972 (+ 3726us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.154.62:36685: Server oryx/127.18.154.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":415,"thread_start_us":336,"threads_started":1}
W20260807 18:13:40.672560 32367 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:13:40.662790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.2:42925 (local address 127.18.154.62:36685)
0807 18:13:40.663044 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:40.663052 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:40.663419 (+ 367us) server_negotiation.cc:407] Connection header received
0807 18:13:40.663562 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:40.663566 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:40.663658 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:40.663749 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:40.664572 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:40.665479 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:40.666812 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:40.667330 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:40.667428 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:40.672273 (+ 4845us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.154.2:42925: BlockingRecv error: recv got EOF from 127.18.154.2:42925 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
Aug 07 18:13:41 dist-test-slave-gkw7 krb5kdc[30876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.154.3@KRBTEST.COM for oryx/127.18.154.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:13:41 dist-test-slave-gkw7 krb5kdc[30876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.154.3@KRBTEST.COM for oryx/127.18.154.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:13:41.106827 32405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:41.097323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:36685 (local address 127.18.154.3:45231)
0807 18:13:41.097952 (+ 629us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:41.097969 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 18:13:41.098091 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:13:41.098385 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:13:41.098388 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:13:41.098819 (+ 431us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:13:41.098988 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:41.098996 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:41.100017 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:41.100021 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:41.101020 (+ 999us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:41.101033 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:41.101938 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:41.101942 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:41.102093 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:41.102824 (+ 731us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:13:41.102857 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:13:41.106572 (+ 3715us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.154.62:36685: Server oryx/127.18.154.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":350,"threads_started":1}
W20260807 18:13:41.107025 32367 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:13:41.097390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.3:45231 (local address 127.18.154.62:36685)
0807 18:13:41.097615 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:41.097620 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:41.098116 (+ 496us) server_negotiation.cc:407] Connection header received
0807 18:13:41.098178 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:41.098182 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:41.098247 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:41.098341 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:41.099136 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:41.099887 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:41.101172 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:41.101735 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:41.101857 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:41.106783 (+ 4926us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.154.3:45231: BlockingRecv error: recv got EOF from 127.18.154.3:45231 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Aug 07 18:13:41 dist-test-slave-gkw7 krb5kdc[30876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.154.1@KRBTEST.COM for oryx/127.18.154.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:13:41 dist-test-slave-gkw7 krb5kdc[30876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.154.1@KRBTEST.COM for oryx/127.18.154.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:13:41.297289 32406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:41.287702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:36685 (local address 127.18.154.1:37977)
0807 18:13:41.288314 (+ 612us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:41.288335 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 18:13:41.288493 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:13:41.288864 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:13:41.288868 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:13:41.289321 (+ 453us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:13:41.289481 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:41.289489 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:41.290584 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:41.290589 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:41.291582 (+ 993us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:41.291593 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:41.292295 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:41.292301 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:41.292483 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:41.293198 (+ 715us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:13:41.293216 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:13:41.296927 (+ 3711us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.154.62:36685: Server oryx/127.18.154.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":494,"thread_start_us":312,"threads_started":1}
W20260807 18:13:41.297363 32367 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:13:41.287793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.1:37977 (local address 127.18.154.62:36685)
0807 18:13:41.288079 (+ 286us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:41.288088 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:41.288425 (+ 337us) server_negotiation.cc:407] Connection header received
0807 18:13:41.288616 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:41.288622 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:41.288709 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:41.288814 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:41.289669 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:41.290442 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:41.291725 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:41.292138 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:41.292278 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:41.297121 (+ 4843us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.154.1:37977: BlockingRecv error: recv got EOF from 127.18.154.1:37977 (error 108)
Metrics: {"server-negotiator.queue_time_us":90}
Aug 07 18:13:41 dist-test-slave-gkw7 krb5kdc[30876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.154.2@KRBTEST.COM for oryx/127.18.154.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:13:41 dist-test-slave-gkw7 krb5kdc[30876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.154.2@KRBTEST.COM for oryx/127.18.154.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:13:41.683039 32408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:41.673754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:36685 (local address 127.18.154.2:58179)
0807 18:13:41.674289 (+ 535us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:41.674307 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 18:13:41.674422 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:13:41.674775 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:13:41.674778 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:13:41.675223 (+ 445us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:13:41.675380 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:41.675389 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:41.676504 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:41.676508 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:41.677508 (+ 1000us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:41.677519 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:41.678328 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:41.678333 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:41.678461 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:41.679149 (+ 688us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:13:41.679167 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:13:41.682747 (+ 3580us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.154.62:36685: Server oryx/127.18.154.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":425,"thread_start_us":284,"threads_started":1}
W20260807 18:13:41.683313 32367 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:13:41.673763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.2:58179 (local address 127.18.154.62:36685)
0807 18:13:41.674008 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:41.674019 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:41.674410 (+ 391us) server_negotiation.cc:407] Connection header received
0807 18:13:41.674531 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:41.674535 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:41.674601 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:41.674681 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:41.675527 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:41.676353 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:41.677708 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:41.678184 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:41.678279 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:41.683007 (+ 4728us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.154.2:58179: BlockingRecv error: recv got EOF from 127.18.154.2:58179 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Aug 07 18:13:42 dist-test-slave-gkw7 krb5kdc[30876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.154.3@KRBTEST.COM for oryx/127.18.154.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:13:42 dist-test-slave-gkw7 krb5kdc[30876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.154.3@KRBTEST.COM for oryx/127.18.154.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:13:42.117650 32409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:42.107943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:36685 (local address 127.18.154.3:36627)
0807 18:13:42.108560 (+ 617us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:42.108577 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 18:13:42.108754 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:13:42.109116 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:13:42.109120 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:13:42.109660 (+ 540us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:13:42.109811 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:42.109820 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:42.110871 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:42.110879 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:42.111862 (+ 983us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:42.111873 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:42.112770 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:42.112776 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:42.112961 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:42.113674 (+ 713us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:13:42.113693 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:13:42.117264 (+ 3571us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.154.62:36685: Server oryx/127.18.154.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":526,"thread_start_us":370,"threads_started":1}
W20260807 18:13:42.117724 32367 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:13:42.108003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.3:36627 (local address 127.18.154.62:36685)
0807 18:13:42.108319 (+ 316us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:42.108328 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:42.108683 (+ 355us) server_negotiation.cc:407] Connection header received
0807 18:13:42.108887 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:42.108899 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:42.108966 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:42.109088 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:42.109959 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:42.110724 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:42.112012 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:42.112587 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:42.112755 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:42.117479 (+ 4724us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.154.3:36627: BlockingRecv error: recv got EOF from 127.18.154.3:36627 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Aug 07 18:13:42 dist-test-slave-gkw7 krb5kdc[30876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.154.1@KRBTEST.COM for oryx/127.18.154.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:13:42 dist-test-slave-gkw7 krb5kdc[30876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.154.1@KRBTEST.COM for oryx/127.18.154.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:13:42.307578 32410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:42.298350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:36685 (local address 127.18.154.1:56351)
0807 18:13:42.298888 (+ 538us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:42.298909 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 18:13:42.299085 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:13:42.299477 (+ 392us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:13:42.299481 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:13:42.299895 (+ 414us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:13:42.300051 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:42.300059 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:42.301170 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:42.301175 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:42.302220 (+ 1045us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:42.302232 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:42.302990 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:42.302997 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:42.303181 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:42.303797 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:13:42.303814 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:13:42.307327 (+ 3513us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.154.62:36685: Server oryx/127.18.154.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":408,"thread_start_us":291,"threads_started":1}
W20260807 18:13:42.307816 32367 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:13:42.298415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.1:56351 (local address 127.18.154.62:36685)
0807 18:13:42.298659 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:42.298665 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:42.299009 (+ 344us) server_negotiation.cc:407] Connection header received
0807 18:13:42.299246 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:42.299252 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:42.299335 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:42.299441 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:42.300226 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:42.301012 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:42.302365 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:42.302829 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:42.302976 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:42.307522 (+ 4546us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.154.1:56351: BlockingRecv error: recv got EOF from 127.18.154.1:56351 (error 108)
Metrics: {"server-negotiator.queue_time_us":91}
Aug 07 18:13:42 dist-test-slave-gkw7 krb5kdc[30876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.154.2@KRBTEST.COM for oryx/127.18.154.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:13:42 dist-test-slave-gkw7 krb5kdc[30876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.154.2@KRBTEST.COM for oryx/127.18.154.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:13:42.694110 32411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:42.684119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:36685 (local address 127.18.154.2:33809)
0807 18:13:42.684636 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:42.684670 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0807 18:13:42.684797 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:13:42.685186 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:13:42.685189 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:13:42.685605 (+ 416us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:13:42.685778 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:42.685786 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:42.686923 (+ 1137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:42.686927 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:42.687968 (+ 1041us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:42.687980 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:42.688873 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:42.688877 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:42.689056 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:42.689984 (+ 928us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:13:42.690010 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:13:42.693865 (+ 3855us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.154.62:36685: Server oryx/127.18.154.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":408,"thread_start_us":296,"threads_started":1}
W20260807 18:13:42.694341 32367 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:13:42.684185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.2:33809 (local address 127.18.154.62:36685)
0807 18:13:42.684437 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:42.684444 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:42.684778 (+ 334us) server_negotiation.cc:407] Connection header received
0807 18:13:42.684922 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:42.684926 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:42.684997 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:42.685175 (+ 178us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:42.685903 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:42.686791 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:42.688130 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:42.688705 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:42.688840 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:42.694096 (+ 5256us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.154.2:33809: BlockingRecv error: recv got EOF from 127.18.154.2:33809 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Aug 07 18:13:43 dist-test-slave-gkw7 krb5kdc[30876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.154.3@KRBTEST.COM for oryx/127.18.154.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:13:43 dist-test-slave-gkw7 krb5kdc[30876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.154.3@KRBTEST.COM for oryx/127.18.154.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:13:43.128705 32415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:43.118675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:36685 (local address 127.18.154.3:44819)
0807 18:13:43.119309 (+ 634us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:43.119328 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 18:13:43.119451 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:13:43.119786 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:13:43.119789 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:13:43.120180 (+ 391us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:13:43.120364 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:43.120373 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:43.121659 (+ 1286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:43.121664 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:43.122747 (+ 1083us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:43.122760 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:43.123572 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:43.123577 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:43.123707 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:43.124393 (+ 686us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:13:43.124412 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:13:43.128318 (+ 3906us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.154.62:36685: Server oryx/127.18.154.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":498,"thread_start_us":344,"threads_started":1}
W20260807 18:13:43.128952 32367 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:13:43.118914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.3:44819 (local address 127.18.154.62:36685)
0807 18:13:43.119198 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:43.119203 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:43.119413 (+ 210us) server_negotiation.cc:407] Connection header received
0807 18:13:43.119557 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:43.119561 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:43.119634 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:43.119737 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:43.120508 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:43.121433 (+ 925us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:43.122900 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:43.123429 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:43.123536 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:43.128629 (+ 5093us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.154.3:44819: BlockingRecv error: recv got EOF from 127.18.154.3:44819 (error 108)
Metrics: {"server-negotiator.queue_time_us":113}
Aug 07 18:13:43 dist-test-slave-gkw7 krb5kdc[30876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.154.1@KRBTEST.COM for oryx/127.18.154.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:13:43 dist-test-slave-gkw7 krb5kdc[30876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.154.1@KRBTEST.COM for oryx/127.18.154.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:13:43.318789 32416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:43.308602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:36685 (local address 127.18.154.1:57661)
0807 18:13:43.309119 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:43.309141 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 18:13:43.309277 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:13:43.309639 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:13:43.309643 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:13:43.310086 (+ 443us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:13:43.310280 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:43.310288 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:43.311297 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:43.311301 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:43.312755 (+ 1454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:43.312770 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:43.313794 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:43.313800 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:43.313946 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:43.314637 (+ 691us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:13:43.314656 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:13:43.318513 (+ 3857us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.154.62:36685: Server oryx/127.18.154.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":394,"thread_start_us":281,"threads_started":1}
W20260807 18:13:43.319005 32367 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:13:43.308784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.1:57661 (local address 127.18.154.62:36685)
0807 18:13:43.309035 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:43.309041 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:43.309236 (+ 195us) server_negotiation.cc:407] Connection header received
0807 18:13:43.309379 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:43.309382 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:43.309446 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:43.309546 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:43.310407 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:43.311177 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:43.312914 (+ 1737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:43.313615 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:43.313722 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:43.318741 (+ 5019us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.154.1:57661: BlockingRecv error: recv got EOF from 127.18.154.1:57661 (error 108)
Metrics: {"server-negotiator.queue_time_us":95}
Aug 07 18:13:43 dist-test-slave-gkw7 krb5kdc[30876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.154.2@KRBTEST.COM for oryx/127.18.154.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:13:43 dist-test-slave-gkw7 krb5kdc[30876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.154.2@KRBTEST.COM for oryx/127.18.154.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:13:43.709959 32417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:43.695232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:36685 (local address 127.18.154.2:54043)
0807 18:13:43.695783 (+ 551us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:43.695802 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 18:13:43.695966 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:13:43.696334 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:13:43.696339 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:13:43.696840 (+ 501us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:13:43.697023 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:43.697032 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:43.698171 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:43.698178 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:43.699132 (+ 954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:43.699145 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:43.699852 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:43.699858 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:43.700048 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:43.705277 (+ 5229us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:13:43.705298 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:13:43.709728 (+ 4430us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.154.62:36685: Server oryx/127.18.154.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":419,"thread_start_us":276,"threads_started":1}
W20260807 18:13:43.710189 32367 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:13:43.695302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.2:54043 (local address 127.18.154.62:36685)
0807 18:13:43.695510 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:43.695518 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:43.695900 (+ 382us) server_negotiation.cc:407] Connection header received
0807 18:13:43.696108 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:43.696115 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:43.696189 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:43.696298 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:43.697185 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:43.698036 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:43.699280 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:43.699695 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:43.699836 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:43.709910 (+ 10074us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.154.2:54043: BlockingRecv error: recv got EOF from 127.18.154.2:54043 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 07 18:13:44 dist-test-slave-gkw7 krb5kdc[30876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.154.3@KRBTEST.COM for oryx/127.18.154.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:13:44 dist-test-slave-gkw7 krb5kdc[30876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.154.3@KRBTEST.COM for oryx/127.18.154.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:13:44.140300 32418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:44.129829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:36685 (local address 127.18.154.3:47253)
0807 18:13:44.130496 (+ 667us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:44.130513 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 18:13:44.130633 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:13:44.130931 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:13:44.130934 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:13:44.131428 (+ 494us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:13:44.131688 (+ 260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:44.131698 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:44.132706 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:44.132711 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:44.133872 (+ 1161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:44.133885 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:44.134841 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:44.134853 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:44.134991 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:44.135717 (+ 726us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:13:44.135735 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:13:44.139974 (+ 4239us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.154.62:36685: Server oryx/127.18.154.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":547,"thread_start_us":445,"threads_started":1}
W20260807 18:13:44.140388 32367 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:13:44.129905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.3:47253 (local address 127.18.154.62:36685)
0807 18:13:44.130107 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:44.130115 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:44.130592 (+ 477us) server_negotiation.cc:407] Connection header received
0807 18:13:44.130733 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:44.130736 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:44.130803 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:44.130890 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:44.131819 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:44.132555 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:44.134021 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:44.134683 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:44.134801 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:44.140162 (+ 5361us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.154.3:47253: BlockingRecv error: recv got EOF from 127.18.154.3:47253 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 07 18:13:44 dist-test-slave-gkw7 krb5kdc[30876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.154.1@KRBTEST.COM for oryx/127.18.154.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:13:44 dist-test-slave-gkw7 krb5kdc[30876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.154.1@KRBTEST.COM for oryx/127.18.154.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:13:44.329564 32419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:44.319847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:36685 (local address 127.18.154.1:55507)
0807 18:13:44.320668 (+ 821us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:44.320695 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0807 18:13:44.320846 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:13:44.321304 (+ 458us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:13:44.321309 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:13:44.321893 (+ 584us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:13:44.322079 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:44.322088 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:44.323144 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:44.323148 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:44.324115 (+ 967us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:44.324126 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:44.324874 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:44.324878 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:44.325012 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:44.325659 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:13:44.325677 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:13:44.329261 (+ 3584us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.154.62:36685: Server oryx/127.18.154.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":661,"thread_start_us":533,"threads_started":1}
W20260807 18:13:44.329897 32367 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:13:44.319979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.1:55507 (local address 127.18.154.62:36685)
0807 18:13:44.320306 (+ 327us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:44.320316 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:44.320785 (+ 469us) server_negotiation.cc:407] Connection header received
0807 18:13:44.320979 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:44.320985 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:44.321095 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:44.321261 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:44.322217 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:44.323009 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:44.324247 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:44.324736 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:44.324842 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:44.329509 (+ 4667us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.154.1:55507: BlockingRecv error: recv got EOF from 127.18.154.1:55507 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Aug 07 18:13:44 dist-test-slave-gkw7 krb5kdc[30876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.154.2@KRBTEST.COM for oryx/127.18.154.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:13:44 dist-test-slave-gkw7 krb5kdc[30876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.154.2@KRBTEST.COM for oryx/127.18.154.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:13:44.721304 32421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:44.711158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:36685 (local address 127.18.154.2:42235)
0807 18:13:44.711774 (+ 616us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:44.711792 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 18:13:44.711980 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:13:44.712501 (+ 521us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:13:44.712504 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:13:44.712939 (+ 435us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:13:44.713252 (+ 313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:44.713270 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:44.714411 (+ 1141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:44.714416 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:44.715419 (+ 1003us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:44.715437 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:44.716201 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:44.716207 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:44.716408 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:44.717060 (+ 652us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:13:44.717079 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:13:44.720944 (+ 3865us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.154.62:36685: Server oryx/127.18.154.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":471,"thread_start_us":342,"threads_started":1}
W20260807 18:13:44.721434 32367 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:13:44.711320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.2:42235 (local address 127.18.154.62:36685)
0807 18:13:44.711552 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:44.711561 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:44.711930 (+ 369us) server_negotiation.cc:407] Connection header received
0807 18:13:44.712222 (+ 292us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:44.712229 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:44.712325 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:44.712443 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:44.713434 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:44.714261 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:44.715579 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:44.716048 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:44.716188 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:44.721194 (+ 5006us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.154.2:42235: BlockingRecv error: recv got EOF from 127.18.154.2:42235 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
Aug 07 18:13:45 dist-test-slave-gkw7 krb5kdc[30876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.154.3@KRBTEST.COM for oryx/127.18.154.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:13:45 dist-test-slave-gkw7 krb5kdc[30876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.154.3@KRBTEST.COM for oryx/127.18.154.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:13:45.151439 32422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:45.141405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:36685 (local address 127.18.154.3:40755)
0807 18:13:45.142182 (+ 777us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:45.142201 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 18:13:45.142365 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:13:45.142743 (+ 378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:13:45.142748 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:13:45.143304 (+ 556us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:13:45.143540 (+ 236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:45.143551 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:45.144758 (+ 1207us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:45.144764 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:45.145795 (+ 1031us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:45.145809 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:45.146555 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:45.146561 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:45.146753 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:45.147380 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:13:45.147398 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:13:45.151086 (+ 3688us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.154.62:36685: Server oryx/127.18.154.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":613,"thread_start_us":455,"threads_started":1}
W20260807 18:13:45.151623 32367 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:13:45.141497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.3:40755 (local address 127.18.154.62:36685)
0807 18:13:45.141730 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:45.141737 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:45.142298 (+ 561us) server_negotiation.cc:407] Connection header received
0807 18:13:45.142493 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:45.142497 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:45.142559 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:45.142673 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:45.143703 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:45.144610 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:45.145936 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:45.146367 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:45.146507 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:45.151335 (+ 4828us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.154.3:40755: BlockingRecv error: recv got EOF from 127.18.154.3:40755 (error 108)
Metrics: {"server-negotiator.queue_time_us":100}
Aug 07 18:13:45 dist-test-slave-gkw7 krb5kdc[30876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.154.1@KRBTEST.COM for oryx/127.18.154.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:13:45 dist-test-slave-gkw7 krb5kdc[30876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.154.1@KRBTEST.COM for oryx/127.18.154.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:13:45.340562 32423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:45.330658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:36685 (local address 127.18.154.1:48917)
0807 18:13:45.331603 (+ 945us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:45.331641 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0807 18:13:45.331768 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:13:45.332119 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:13:45.332122 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:13:45.332585 (+ 463us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:13:45.332765 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:45.332775 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:45.333782 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:45.333787 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:45.334869 (+ 1082us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:45.334883 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:45.335587 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:45.335592 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:45.335738 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:45.336417 (+ 679us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:13:45.336437 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:13:45.340325 (+ 3888us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.154.62:36685: Server oryx/127.18.154.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":762,"thread_start_us":628,"threads_started":1}
W20260807 18:13:45.340819 32367 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:13:45.330820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.1:48917 (local address 127.18.154.62:36685)
0807 18:13:45.331107 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:45.331117 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:45.331730 (+ 613us) server_negotiation.cc:407] Connection header received
0807 18:13:45.331874 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:45.331877 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:45.331941 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:45.332025 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:45.332896 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:45.333648 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:45.335021 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:45.335431 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:45.335541 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:45.340558 (+ 5017us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.154.1:48917: BlockingRecv error: recv got EOF from 127.18.154.1:48917 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Aug 07 18:13:45 dist-test-slave-gkw7 krb5kdc[30876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.154.2@KRBTEST.COM for oryx/127.18.154.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:13:45 dist-test-slave-gkw7 krb5kdc[30876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.154.2@KRBTEST.COM for oryx/127.18.154.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:13:45.732263 32424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:45.722440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:36685 (local address 127.18.154.2:34121)
0807 18:13:45.722970 (+ 530us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:45.722987 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 18:13:45.723115 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:13:45.723451 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:13:45.723454 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:13:45.723898 (+ 444us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:13:45.724046 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:45.724055 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:45.725350 (+ 1295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:45.725354 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:45.726428 (+ 1074us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:45.726442 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:45.727213 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:45.727217 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:45.727347 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:45.727950 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:13:45.727967 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:13:45.731915 (+ 3948us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.154.62:36685: Server oryx/127.18.154.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":431,"thread_start_us":323,"threads_started":1}
W20260807 18:13:45.732630 32367 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:13:45.722659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.2:34121 (local address 127.18.154.62:36685)
0807 18:13:45.722927 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:45.722935 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:45.723069 (+ 134us) server_negotiation.cc:407] Connection header received
0807 18:13:45.723243 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:45.723247 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:45.723315 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:45.723401 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:45.724220 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:45.725173 (+ 953us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:45.726587 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:45.727078 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:45.727180 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:45.732173 (+ 4993us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.154.2:34121: BlockingRecv error: recv got EOF from 127.18.154.2:34121 (error 108)
Metrics: {"server-negotiator.queue_time_us":102}
Aug 07 18:13:46 dist-test-slave-gkw7 krb5kdc[30876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.154.3@KRBTEST.COM for oryx/127.18.154.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:13:46 dist-test-slave-gkw7 krb5kdc[30876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.154.3@KRBTEST.COM for oryx/127.18.154.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:13:46.162765 32425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:46.152588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:36685 (local address 127.18.154.3:33749)
0807 18:13:46.153083 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:46.153098 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 18:13:46.153314 (+ 216us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:13:46.153700 (+ 386us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:13:46.153704 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:13:46.154214 (+ 510us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:13:46.154405 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:46.154418 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:46.155797 (+ 1379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:46.155804 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:46.156928 (+ 1124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:46.156941 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:46.157822 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:46.157827 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:46.157959 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:46.158665 (+ 706us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:13:46.158684 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:13:46.162417 (+ 3733us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.154.62:36685: Server oryx/127.18.154.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":405,"thread_start_us":280,"threads_started":1}
W20260807 18:13:46.162876 32367 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:13:46.152644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.3:33749 (local address 127.18.154.62:36685)
0807 18:13:46.152878 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:46.152886 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:46.153225 (+ 339us) server_negotiation.cc:407] Connection header received
0807 18:13:46.153444 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:46.153453 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:46.153517 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:46.153664 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:46.154579 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:46.155650 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:46.157130 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:46.157682 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:46.157790 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:46.162640 (+ 4850us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.154.3:33749: BlockingRecv error: recv got EOF from 127.18.154.3:33749 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 07 18:13:46 dist-test-slave-gkw7 krb5kdc[30876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.154.1@KRBTEST.COM for oryx/127.18.154.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:13:46 dist-test-slave-gkw7 krb5kdc[30876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.154.1@KRBTEST.COM for oryx/127.18.154.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:13:46.351933 32426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:46.341948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:36685 (local address 127.18.154.1:60703)
0807 18:13:46.342543 (+ 595us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:46.342565 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 18:13:46.342752 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:13:46.343123 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:13:46.343128 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:13:46.343630 (+ 502us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:13:46.343829 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:46.343839 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:46.344942 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:46.344948 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:46.345955 (+ 1007us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:46.345967 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:46.346754 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:46.346759 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:46.346953 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:46.347630 (+ 677us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:13:46.347649 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:13:46.351654 (+ 4005us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.154.62:36685: Server oryx/127.18.154.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":469,"thread_start_us":331,"threads_started":1}
W20260807 18:13:46.352074 32367 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:13:46.342022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.1:60703 (local address 127.18.154.62:36685)
0807 18:13:46.342414 (+ 392us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:46.342421 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:46.342672 (+ 251us) server_negotiation.cc:407] Connection header received
0807 18:13:46.342898 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:46.342903 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:46.342979 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:46.343099 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:46.343981 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:46.344772 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:46.346111 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:46.346601 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:46.346740 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:46.351837 (+ 5097us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.154.1:60703: BlockingRecv error: recv got EOF from 127.18.154.1:60703 (error 108)
Metrics: {"server-negotiator.queue_time_us":249}
Aug 07 18:13:46 dist-test-slave-gkw7 krb5kdc[30876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.154.2@KRBTEST.COM for oryx/127.18.154.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:13:46 dist-test-slave-gkw7 krb5kdc[30876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.154.2@KRBTEST.COM for oryx/127.18.154.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:13:46.743904 32427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:46.733881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:36685 (local address 127.18.154.2:37409)
0807 18:13:46.734480 (+ 599us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:46.734498 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 18:13:46.734623 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:13:46.734946 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:13:46.734950 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:13:46.735403 (+ 453us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:13:46.735612 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:46.735622 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:46.736632 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:46.736636 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:46.737850 (+ 1214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:46.737864 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:46.738706 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:46.738711 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:46.738868 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:46.739668 (+ 800us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:13:46.739689 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:13:46.743562 (+ 3873us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.154.62:36685: Server oryx/127.18.154.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":451,"thread_start_us":316,"threads_started":1}
W20260807 18:13:46.744163 32367 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:13:46.733927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.2:37409 (local address 127.18.154.62:36685)
0807 18:13:46.734101 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:46.734107 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:46.734582 (+ 475us) server_negotiation.cc:407] Connection header received
0807 18:13:46.734736 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:46.734739 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:46.734804 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:46.734889 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:46.735762 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:46.736471 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:46.738018 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:46.738532 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:46.738667 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:46.743833 (+ 5166us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.154.2:37409: BlockingRecv error: recv got EOF from 127.18.154.2:37409 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 18:13:46.818796 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 31760
I20260807 18:13:46.833460 32436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:46.820397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34964 (local address 127.18.154.1:44115)
0807 18:13:46.821119 (+ 722us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:46.821126 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:46.821194 (+ 68us) server_negotiation.cc:407] Connection header received
0807 18:13:46.821318 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:46.821330 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:46.821654 (+ 324us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:46.821896 (+ 242us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:46.823530 (+ 1634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:46.824656 (+ 1126us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:46.826022 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:46.826618 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:46.826776 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:46.829655 (+ 2879us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:13:46.829666 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:13:46.829682 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:13:46.829771 (+ 89us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:13:46.832686 (+ 2915us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.18.154.1@KRBTEST.COM not found in keytab (ticket kvno 2)
0807 18:13:46.833099 (+ 413us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:34964: Request ticket server kudu/127.18.154.1@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":535,"thread_start_us":397,"threads_started":1}
W20260807 18:13:46.833609 32436 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:34964: Request ticket server kudu/127.18.154.1@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260807 18:13:46.833580 19048 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 31760: Not authorized: Client connection negotiation failed: client connection to 127.18.154.1:44115: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.18.154.1@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260807 18:13:46.833843 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 31760
I20260807 18:13:46.858575 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 31995
I20260807 18:13:46.871991 32437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:46.859877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34934 (local address 127.18.154.2:36957)
0807 18:13:46.860592 (+ 715us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:46.860600 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:46.860629 (+ 29us) server_negotiation.cc:407] Connection header received
0807 18:13:46.860701 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:46.860707 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:46.860951 (+ 244us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:46.861185 (+ 234us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:46.862629 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:46.863739 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:46.865054 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:46.865820 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:46.865964 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:46.868457 (+ 2493us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:13:46.868467 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:13:46.868482 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:13:46.868526 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:13:46.871260 (+ 2734us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.18.154.2@KRBTEST.COM not found in keytab (ticket kvno 2)
0807 18:13:46.871641 (+ 381us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:34934: Request ticket server kudu/127.18.154.2@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":531,"thread_start_us":393,"threads_started":1}
W20260807 18:13:46.872048 19048 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 31995: Not authorized: Client connection negotiation failed: client connection to 127.18.154.2:36957: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.18.154.2@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260807 18:13:46.872126 32437 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:34934: Request ticket server kudu/127.18.154.2@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260807 18:13:46.872216 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 31995
I20260807 18:13:46.897810 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 32181
I20260807 18:13:46.911242 32439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:46.899147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45504 (local address 127.18.154.3:34321)
0807 18:13:46.899912 (+ 765us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:46.899920 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:46.899947 (+ 27us) server_negotiation.cc:407] Connection header received
0807 18:13:46.900018 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:46.900024 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:46.900287 (+ 263us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:46.900517 (+ 230us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:46.902069 (+ 1552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:46.903111 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:46.904364 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:46.904928 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:46.905080 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:46.907764 (+ 2684us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:13:46.907774 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:13:46.907788 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:13:46.907836 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:13:46.910506 (+ 2670us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.18.154.3@KRBTEST.COM not found in keytab (ticket kvno 2)
0807 18:13:46.910870 (+ 364us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:45504: Request ticket server kudu/127.18.154.3@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":550,"thread_start_us":416,"threads_started":1}
W20260807 18:13:46.911245 19048 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 32181: Not authorized: Client connection negotiation failed: client connection to 127.18.154.3:34321: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.18.154.3@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260807 18:13:46.911362 32439 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:45504: Request ticket server kudu/127.18.154.3@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260807 18:13:46.911396 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 32181
I20260807 18:13:46.935503 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 31653
I20260807 18:13:46.937160 31752 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:13:47.057356 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 31653
2026-08-07T18:13:47Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (21912 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-07T18:13:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:13:47Z Disabled control of system clock
I20260807 18:13:47.152896 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126328808105-19048-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126328808105-19048-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126328808105-19048-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.154.62:39167
--webserver_interface=127.18.154.62
--webserver_port=0
--builtin_ntp_servers=127.18.154.20:37231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.154.62:39167
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260807 18:13:47.524479 32445 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:13:47.524820 32445 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:13:47.524912 32445 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:13:47.533710 32445 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:13:47.533845 32445 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:13:47.533901 32445 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:13:47.533969 32445 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:13:47.534027 32445 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:13:47.543798 32445 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.154.20:37231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126328808105-19048-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126328808105-19048-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.154.62:39167
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.154.62:39167
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.154.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126328808105-19048-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:13:47.545176 32445 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:13:47.547129 32445 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:13:47.559806 32454 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:13:47.559906 32452 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:13:47.559878 32451 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:13:47.560714 32445 server_base.cc:1034] running on GCE node
I20260807 18:13:47.561684 32445 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:13:47.563202 32445 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:13:47.564451 32445 hybrid_clock.cc:648] HybridClock initialized: now 1786126427564400 us; error 32 us; skew 500 ppm
I20260807 18:13:47.564877 32445 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 18:13:47.567302 32445 webserver.cc:460] Webserver started at http://127.18.154.62:39877/ using document root <none> and password file <none>
I20260807 18:13:47.568040 32445 fs_manager.cc:359] Metadata directory not provided
I20260807 18:13:47.568161 32445 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:13:47.568514 32445 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:13:47.571532 32445 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126328808105-19048-0/minicluster-data/master-0/data/instance:
uuid: "d07852dbe87a406c97ac2937ef1708d7"
format_stamp: "Formatted at 2026-08-07 18:13:47 on dist-test-slave-gkw7"
server_key: "a7878144b50d9e1603049ba7aa2ee21d"
server_key_iv: "eb02a18720a2ed04d81d9b3da4a5b86d"
server_key_version: "encryptionkey@0"
I20260807 18:13:47.572369 32445 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126328808105-19048-0/minicluster-data/master-0/wal/instance:
uuid: "d07852dbe87a406c97ac2937ef1708d7"
format_stamp: "Formatted at 2026-08-07 18:13:47 on dist-test-slave-gkw7"
server_key: "a7878144b50d9e1603049ba7aa2ee21d"
server_key_iv: "eb02a18720a2ed04d81d9b3da4a5b86d"
server_key_version: "encryptionkey@0"
I20260807 18:13:47.577930 32445 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260807 18:13:47.581714 32463 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:13:47.583249 32445 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 18:13:47.583449 32445 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126328808105-19048-0/minicluster-data/master-0/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126328808105-19048-0/minicluster-data/master-0/wal
uuid: "d07852dbe87a406c97ac2937ef1708d7"
format_stamp: "Formatted at 2026-08-07 18:13:47 on dist-test-slave-gkw7"
server_key: "a7878144b50d9e1603049ba7aa2ee21d"
server_key_iv: "eb02a18720a2ed04d81d9b3da4a5b86d"
server_key_version: "encryptionkey@0"
I20260807 18:13:47.583703 32445 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126328808105-19048-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126328808105-19048-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126328808105-19048-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:13:47.599222 32445 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:13:47.600175 32445 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:13:47.600502 32445 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:13:47.620637 32445 rpc_server.cc:307] RPC server started. Bound to: 127.18.154.62:39167
I20260807 18:13:47.620674 32547 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.154.62:39167 every 8 connection(s)
I20260807 18:13:47.622164 32445 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
I20260807 18:13:47.623690 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 32445
I20260807 18:13:47.623929 19048 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126328808105-19048-0/minicluster-data/master-0/wal/instance
I20260807 18:13:47.624572 19048 external_mini_cluster.cc:1428] Setting key 8dadab6e9f27b43c292eb18d8004c837
I20260807 18:13:47.627352 32548 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:13:47.642460 32552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:47.626822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34846 (local address 127.18.154.62:39167)
0807 18:13:47.628630 (+ 1808us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:47.628647 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:47.628704 (+ 57us) server_negotiation.cc:407] Connection header received
0807 18:13:47.630989 (+ 2285us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:47.631020 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:47.631689 (+ 669us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:47.632192 (+ 503us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:47.632829 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:47.635028 (+ 2199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:47.636557 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:47.637709 (+ 1152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:47.637929 (+ 220us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:47.639132 (+ 1203us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:13:47.639144 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:13:47.639165 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:13:47.640027 (+ 862us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:13:47.640143 (+ 116us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:13:47.640185 (+ 42us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:13:47.640312 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0807 18:13:47.640563 (+ 251us) server_negotiation.cc:299] Negotiation successful
0807 18:13:47.640945 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1246,"thread_start_us":938,"threads_started":1}
I20260807 18:13:47.643710 32548 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d07852dbe87a406c97ac2937ef1708d7: Bootstrap starting.
I20260807 18:13:47.647701 32548 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d07852dbe87a406c97ac2937ef1708d7: Neither blocks nor log segments found. Creating new log.
I20260807 18:13:47.648850 32548 log.cc:824] T 00000000000000000000000000000000 P d07852dbe87a406c97ac2937ef1708d7: Log is configured to *not* fsync() on all Append() calls
I20260807 18:13:47.652251 32548 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d07852dbe87a406c97ac2937ef1708d7: No bootstrap required, opened a new log
I20260807 18:13:47.659935 32548 raft_consensus.cc:348] T 00000000000000000000000000000000 P d07852dbe87a406c97ac2937ef1708d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d07852dbe87a406c97ac2937ef1708d7" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 39167 } }
I20260807 18:13:47.660338 32548 raft_consensus.cc:374] T 00000000000000000000000000000000 P d07852dbe87a406c97ac2937ef1708d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:13:47.660425 32548 raft_consensus.cc:729] T 00000000000000000000000000000000 P d07852dbe87a406c97ac2937ef1708d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d07852dbe87a406c97ac2937ef1708d7, State: Initialized, Role: FOLLOWER
I20260807 18:13:47.661026 32548 consensus_queue.cc:260] T 00000000000000000000000000000000 P d07852dbe87a406c97ac2937ef1708d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d07852dbe87a406c97ac2937ef1708d7" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 39167 } }
I20260807 18:13:47.661247 32548 raft_consensus.cc:388] T 00000000000000000000000000000000 P d07852dbe87a406c97ac2937ef1708d7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:13:47.661347 32548 raft_consensus.cc:482] T 00000000000000000000000000000000 P d07852dbe87a406c97ac2937ef1708d7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:13:47.661480 32548 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d07852dbe87a406c97ac2937ef1708d7 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:13:47.663707 32548 raft_consensus.cc:504] T 00000000000000000000000000000000 P d07852dbe87a406c97ac2937ef1708d7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d07852dbe87a406c97ac2937ef1708d7" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 39167 } }
I20260807 18:13:47.664279 32548 leader_election.cc:302] T 00000000000000000000000000000000 P d07852dbe87a406c97ac2937ef1708d7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d07852dbe87a406c97ac2937ef1708d7; no voters:
I20260807 18:13:47.664834 32548 leader_election.cc:288] T 00000000000000000000000000000000 P d07852dbe87a406c97ac2937ef1708d7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:13:47.665066 32554 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d07852dbe87a406c97ac2937ef1708d7 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:13:47.666214 32554 raft_consensus.cc:686] T 00000000000000000000000000000000 P d07852dbe87a406c97ac2937ef1708d7 [term 1 LEADER]: Becoming Leader. State: Replica: d07852dbe87a406c97ac2937ef1708d7, State: Running, Role: LEADER
I20260807 18:13:47.666812 32554 consensus_queue.cc:237] T 00000000000000000000000000000000 P d07852dbe87a406c97ac2937ef1708d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d07852dbe87a406c97ac2937ef1708d7" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 39167 } }
I20260807 18:13:47.667192 32548 sys_catalog.cc:564] T 00000000000000000000000000000000 P d07852dbe87a406c97ac2937ef1708d7 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:13:47.670240 32557 sys_catalog.cc:455] T 00000000000000000000000000000000 P d07852dbe87a406c97ac2937ef1708d7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d07852dbe87a406c97ac2937ef1708d7. Latest consensus state: current_term: 1 leader_uuid: "d07852dbe87a406c97ac2937ef1708d7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d07852dbe87a406c97ac2937ef1708d7" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 39167 } } }
I20260807 18:13:47.670643 32557 sys_catalog.cc:458] T 00000000000000000000000000000000 P d07852dbe87a406c97ac2937ef1708d7 [sys.catalog]: This master's current role is: LEADER
I20260807 18:13:47.671393 32555 sys_catalog.cc:455] T 00000000000000000000000000000000 P d07852dbe87a406c97ac2937ef1708d7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d07852dbe87a406c97ac2937ef1708d7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d07852dbe87a406c97ac2937ef1708d7" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 39167 } } }
I20260807 18:13:47.671813 32555 sys_catalog.cc:458] T 00000000000000000000000000000000 P d07852dbe87a406c97ac2937ef1708d7 [sys.catalog]: This master's current role is: LEADER
I20260807 18:13:47.674104 32566 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:13:47.678789 32566 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:13:47.687510 32566 catalog_manager.cc:1347] Generated new cluster ID: fe737bfd8338475c8e5d82472efa6431
I20260807 18:13:47.687657 32566 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:13:47.713004 32566 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:13:47.714592 32566 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:13:47.723471 32566 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d07852dbe87a406c97ac2937ef1708d7: Generated new TSK 0
I20260807 18:13:47.724395 32566 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:13:47.735178 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126328808105-19048-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126328808105-19048-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126328808105-19048-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126328808105-19048-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.154.1:0
--local_ip_for_outbound_sockets=127.18.154.1
--webserver_interface=127.18.154.1
--webserver_port=0
--tserver_master_addrs=127.18.154.62:39167
--builtin_ntp_servers=127.18.154.20:37231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260807 18:13:48.106086 32586 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:13:48.106423 32586 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:13:48.106516 32586 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:13:48.114305 32586 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:13:48.114432 32586 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:13:48.114638 32586 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.154.1
I20260807 18:13:48.124847 32586 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.154.20:37231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126328808105-19048-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126328808105-19048-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.154.1:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126328808105-19048-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.154.1
--webserver_port=0
--tserver_master_addrs=127.18.154.62:39167
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.154.1
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126328808105-19048-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:13:48.126305 32586 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:13:48.128253 32586 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:13:48.142886 32596 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:13:48.142843 32597 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:13:48.142797 32599 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:13:48.144274 32586 server_base.cc:1034] running on GCE node
I20260807 18:13:48.145148 32586 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:13:48.146414 32586 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:13:48.147678 32586 hybrid_clock.cc:648] HybridClock initialized: now 1786126428147617 us; error 52 us; skew 500 ppm
I20260807 18:13:48.148058 32586 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 18:13:48.150873 32586 webserver.cc:460] Webserver started at http://127.18.154.1:44955/ using document root <none> and password file <none>
I20260807 18:13:48.151681 32586 fs_manager.cc:359] Metadata directory not provided
I20260807 18:13:48.151834 32586 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:13:48.152227 32586 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:13:48.155515 32586 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126328808105-19048-0/minicluster-data/ts-0/data/instance:
uuid: "670fc1c48b674358a60dc376bd3cb3d2"
format_stamp: "Formatted at 2026-08-07 18:13:48 on dist-test-slave-gkw7"
server_key: "a31ba90589c1673583972ec59b3acb81"
server_key_iv: "6142a085b461d15e511e25b5378a7c06"
server_key_version: "encryptionkey@0"
I20260807 18:13:48.156409 32586 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126328808105-19048-0/minicluster-data/ts-0/wal/instance:
uuid: "670fc1c48b674358a60dc376bd3cb3d2"
format_stamp: "Formatted at 2026-08-07 18:13:48 on dist-test-slave-gkw7"
server_key: "a31ba90589c1673583972ec59b3acb81"
server_key_iv: "6142a085b461d15e511e25b5378a7c06"
server_key_version: "encryptionkey@0"
I20260807 18:13:48.162293 32586 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260807 18:13:48.166189 32610 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:13:48.167596 32586 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 18:13:48.167826 32586 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126328808105-19048-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126328808105-19048-0/minicluster-data/ts-0/wal
uuid: "670fc1c48b674358a60dc376bd3cb3d2"
format_stamp: "Formatted at 2026-08-07 18:13:48 on dist-test-slave-gkw7"
server_key: "a31ba90589c1673583972ec59b3acb81"
server_key_iv: "6142a085b461d15e511e25b5378a7c06"
server_key_version: "encryptionkey@0"
I20260807 18:13:48.168090 32586 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126328808105-19048-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126328808105-19048-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126328808105-19048-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:13:48.183382 32586 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:13:48.184293 32586 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:13:48.184623 32586 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:13:48.185727 32586 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:13:48.187463 32586 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:13:48.187592 32586 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:13:48.187740 32586 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:13:48.187834 32586 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:13:48.223407 32586 rpc_server.cc:307] RPC server started. Bound to: 127.18.154.1:39595
I20260807 18:13:48.223443 307 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.154.1:39595 every 8 connection(s)
I20260807 18:13:48.224831 32586 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126328808105-19048-0/minicluster-data/ts-0/data/info.pb
I20260807 18:13:48.226680 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 32586
I20260807 18:13:48.226912 19048 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126328808105-19048-0/minicluster-data/ts-0/wal/instance
I20260807 18:13:48.227665 19048 external_mini_cluster.cc:1428] Setting key 8931832fa3eb4d1fa9bd04efb110e1ab
I20260807 18:13:48.230942 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126328808105-19048-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126328808105-19048-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126328808105-19048-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126328808105-19048-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.154.2:0
--local_ip_for_outbound_sockets=127.18.154.2
--webserver_interface=127.18.154.2
--webserver_port=0
--tserver_master_addrs=127.18.154.62:39167
--builtin_ntp_servers=127.18.154.20:37231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260807 18:13:48.240069 310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:48.227890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.1:46509 (local address 127.18.154.62:39167)
0807 18:13:48.228359 (+ 469us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:48.228365 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:48.230089 (+ 1724us) server_negotiation.cc:407] Connection header received
0807 18:13:48.232149 (+ 2060us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:48.232157 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:48.232263 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:48.232392 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:48.233730 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:48.234844 (+ 1114us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:48.236549 (+ 1705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:48.237265 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:48.237413 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:48.239000 (+ 1587us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:13:48.239008 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:13:48.239016 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:13:48.239044 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:13:48.239100 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:13:48.239118 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:13:48.239222 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0807 18:13:48.239469 (+ 247us) server_negotiation.cc:299] Negotiation successful
0807 18:13:48.239604 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":340,"thread_start_us":234,"threads_started":1}
I20260807 18:13:48.240988 311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:48.228509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:39167 (local address 127.18.154.1:46509)
0807 18:13:48.229886 (+ 1377us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:48.229953 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0807 18:13:48.231852 (+ 1899us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:13:48.232614 (+ 762us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:13:48.232623 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:13:48.232717 (+ 94us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:13:48.233471 (+ 754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:48.233495 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:48.235025 (+ 1530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:48.235029 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:48.236377 (+ 1348us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:48.236390 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:48.237503 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:48.237508 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:48.237809 (+ 301us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:48.238631 (+ 822us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 18:13:48.238633 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:13:48.238686 (+ 53us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 18:13:48.238776 (+ 90us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:13:48.239227 (+ 451us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:13:48.239236 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:13:48.239243 (+ 7us) client_negotiation.cc:769] Sending connection context
0807 18:13:48.239405 (+ 162us) client_negotiation.cc:240] Negotiation successful
0807 18:13:48.239757 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":715,"thread_start_us":402,"threads_started":1}
I20260807 18:13:48.242833 308 heartbeater.cc:347] Connected to a master server at 127.18.154.62:39167
I20260807 18:13:48.243230 308 heartbeater.cc:464] Registering TS with master...
I20260807 18:13:48.244225 308 heartbeater.cc:511] Master 127.18.154.62:39167 requested a full tablet report, sending...
I20260807 18:13:48.246783 32491 ts_manager.cc:194] Registered new tserver with Master: 670fc1c48b674358a60dc376bd3cb3d2 (127.18.154.1:39595)
I20260807 18:13:48.249058 32491 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.154.1:46509
W20260807 18:13:48.594228 312 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:13:48.594543 312 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:13:48.594630 312 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:13:48.602211 312 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:13:48.602330 312 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:13:48.602439 312 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.154.2
I20260807 18:13:48.612483 312 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.154.20:37231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126328808105-19048-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126328808105-19048-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.154.2:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126328808105-19048-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.154.2
--webserver_port=0
--tserver_master_addrs=127.18.154.62:39167
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.154.2
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126328808105-19048-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:13:48.614046 312 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:13:48.615901 312 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:13:48.631306 321 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:13:48.631549 323 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:13:48.632738 312 server_base.cc:1034] running on GCE node
W20260807 18:13:48.634128 319 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:13:48.634917 312 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:13:48.636159 312 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:13:48.637415 312 hybrid_clock.cc:648] HybridClock initialized: now 1786126428637341 us; error 71 us; skew 500 ppm
I20260807 18:13:48.637830 312 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 18:13:48.640481 312 webserver.cc:460] Webserver started at http://127.18.154.2:44339/ using document root <none> and password file <none>
I20260807 18:13:48.641275 312 fs_manager.cc:359] Metadata directory not provided
I20260807 18:13:48.641407 312 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:13:48.641861 312 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:13:48.645053 312 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126328808105-19048-0/minicluster-data/ts-1/data/instance:
uuid: "ca606b974e254cea873d104f854bbed0"
format_stamp: "Formatted at 2026-08-07 18:13:48 on dist-test-slave-gkw7"
server_key: "26c88b7a318c4227e33c09098a4820ce"
server_key_iv: "9184d980c83844e21cfd1aa83b354670"
server_key_version: "encryptionkey@0"
I20260807 18:13:48.646005 312 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126328808105-19048-0/minicluster-data/ts-1/wal/instance:
uuid: "ca606b974e254cea873d104f854bbed0"
format_stamp: "Formatted at 2026-08-07 18:13:48 on dist-test-slave-gkw7"
server_key: "26c88b7a318c4227e33c09098a4820ce"
server_key_iv: "9184d980c83844e21cfd1aa83b354670"
server_key_version: "encryptionkey@0"
I20260807 18:13:48.651768 312 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 18:13:48.655738 334 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:13:48.657266 312 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 18:13:48.657480 312 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126328808105-19048-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126328808105-19048-0/minicluster-data/ts-1/wal
uuid: "ca606b974e254cea873d104f854bbed0"
format_stamp: "Formatted at 2026-08-07 18:13:48 on dist-test-slave-gkw7"
server_key: "26c88b7a318c4227e33c09098a4820ce"
server_key_iv: "9184d980c83844e21cfd1aa83b354670"
server_key_version: "encryptionkey@0"
I20260807 18:13:48.657799 312 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126328808105-19048-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126328808105-19048-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126328808105-19048-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:13:48.682823 312 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:13:48.683856 312 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:13:48.684280 312 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:13:48.685420 312 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:13:48.687145 312 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:13:48.687337 312 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:13:48.687489 312 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:13:48.687606 312 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:13:48.725519 312 rpc_server.cc:307] RPC server started. Bound to: 127.18.154.2:34621
I20260807 18:13:48.725603 501 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.154.2:34621 every 8 connection(s)
I20260807 18:13:48.727295 312 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126328808105-19048-0/minicluster-data/ts-1/data/info.pb
I20260807 18:13:48.735841 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 312
I20260807 18:13:48.736045 19048 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126328808105-19048-0/minicluster-data/ts-1/wal/instance
I20260807 18:13:48.736595 19048 external_mini_cluster.cc:1428] Setting key 0ce2a1501ba6680dc9162323a0620ae4
I20260807 18:13:48.739923 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126328808105-19048-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126328808105-19048-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126328808105-19048-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126328808105-19048-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.154.3:0
--local_ip_for_outbound_sockets=127.18.154.3
--webserver_interface=127.18.154.3
--webserver_port=0
--tserver_master_addrs=127.18.154.62:39167
--builtin_ntp_servers=127.18.154.20:37231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260807 18:13:48.751883 310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:48.730253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.2:38587 (local address 127.18.154.62:39167)
0807 18:13:48.730456 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:48.730461 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:48.731901 (+ 1440us) server_negotiation.cc:407] Connection header received
0807 18:13:48.733885 (+ 1984us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:48.733889 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:48.733963 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:48.734061 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:48.735432 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:48.736600 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:48.738628 (+ 2028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:48.739186 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:48.739321 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:48.750842 (+ 11521us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:13:48.750851 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:13:48.750858 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:13:48.750890 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:13:48.750943 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:13:48.750964 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:13:48.751083 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0807 18:13:48.751417 (+ 334us) server_negotiation.cc:299] Negotiation successful
0807 18:13:48.751561 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 18:13:48.753466 504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:48.730705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:39167 (local address 127.18.154.2:38587)
0807 18:13:48.731727 (+ 1022us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:48.731766 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 18:13:48.733580 (+ 1814us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:13:48.734381 (+ 801us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:13:48.734390 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:13:48.734483 (+ 93us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:13:48.735119 (+ 636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:48.735137 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:48.736796 (+ 1659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:48.736800 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:48.738465 (+ 1665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:48.738480 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:48.739430 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:48.739435 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:48.739671 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:48.750340 (+ 10669us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 18:13:48.750344 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:13:48.750400 (+ 56us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 18:13:48.750531 (+ 131us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:13:48.751140 (+ 609us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:13:48.751152 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:13:48.751160 (+ 8us) client_negotiation.cc:769] Sending connection context
0807 18:13:48.751335 (+ 175us) client_negotiation.cc:240] Negotiation successful
0807 18:13:48.751906 (+ 571us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":539,"thread_start_us":333,"threads_started":1}
I20260807 18:13:48.755148 502 heartbeater.cc:347] Connected to a master server at 127.18.154.62:39167
I20260807 18:13:48.755502 502 heartbeater.cc:464] Registering TS with master...
I20260807 18:13:48.756573 502 heartbeater.cc:511] Master 127.18.154.62:39167 requested a full tablet report, sending...
I20260807 18:13:48.758246 32491 ts_manager.cc:194] Registered new tserver with Master: ca606b974e254cea873d104f854bbed0 (127.18.154.2:34621)
I20260807 18:13:48.759084 32491 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.154.2:38587
I20260807 18:13:49.252802 308 heartbeater.cc:503] Master 127.18.154.62:39167 was elected leader, sending a full tablet report...
W20260807 18:13:49.302019 505 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:13:49.302724 505 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:13:49.302886 505 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:13:49.319433 505 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:13:49.319628 505 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:13:49.319803 505 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.154.3
I20260807 18:13:49.338518 505 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.154.20:37231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126328808105-19048-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126328808105-19048-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.154.3:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126328808105-19048-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.154.3
--webserver_port=0
--tserver_master_addrs=127.18.154.62:39167
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.154.3
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126328808105-19048-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:13:49.341506 505 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:13:49.345506 505 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:13:49.374862 516 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:13:49.374763 514 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:13:49.375105 513 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:13:49.381908 505 server_base.cc:1034] running on GCE node
I20260807 18:13:49.383417 505 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:13:49.385309 505 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:13:49.388931 505 hybrid_clock.cc:648] HybridClock initialized: now 1786126429388707 us; error 171 us; skew 500 ppm
I20260807 18:13:49.389829 505 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 18:13:49.395143 505 webserver.cc:460] Webserver started at http://127.18.154.3:43563/ using document root <none> and password file <none>
I20260807 18:13:49.397044 505 fs_manager.cc:359] Metadata directory not provided
I20260807 18:13:49.397257 505 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:13:49.398048 505 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:13:49.403954 505 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126328808105-19048-0/minicluster-data/ts-2/data/instance:
uuid: "dc532e1045a54efc9e2732280656fba0"
format_stamp: "Formatted at 2026-08-07 18:13:49 on dist-test-slave-gkw7"
server_key: "b75b3dfd304b7d78d837eb9c7d1986f6"
server_key_iv: "b9e9484a31a092d15a38e5ff5b4cf75e"
server_key_version: "encryptionkey@0"
I20260807 18:13:49.405756 505 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126328808105-19048-0/minicluster-data/ts-2/wal/instance:
uuid: "dc532e1045a54efc9e2732280656fba0"
format_stamp: "Formatted at 2026-08-07 18:13:49 on dist-test-slave-gkw7"
server_key: "b75b3dfd304b7d78d837eb9c7d1986f6"
server_key_iv: "b9e9484a31a092d15a38e5ff5b4cf75e"
server_key_version: "encryptionkey@0"
I20260807 18:13:49.416220 505 fs_manager.cc:691] Time spent creating directory manager: real 0.010s user 0.004s sys 0.004s
I20260807 18:13:49.422793 529 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:13:49.425904 505 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.006s sys 0.000s
I20260807 18:13:49.426265 505 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126328808105-19048-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126328808105-19048-0/minicluster-data/ts-2/wal
uuid: "dc532e1045a54efc9e2732280656fba0"
format_stamp: "Formatted at 2026-08-07 18:13:49 on dist-test-slave-gkw7"
server_key: "b75b3dfd304b7d78d837eb9c7d1986f6"
server_key_iv: "b9e9484a31a092d15a38e5ff5b4cf75e"
server_key_version: "encryptionkey@0"
I20260807 18:13:49.426738 505 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126328808105-19048-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126328808105-19048-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126328808105-19048-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:13:49.462311 505 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:13:49.464514 505 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:13:49.465222 505 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:13:49.467448 505 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:13:49.471540 505 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:13:49.471769 505 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:13:49.472024 505 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:13:49.472241 505 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:13:49.546355 505 rpc_server.cc:307] RPC server started. Bound to: 127.18.154.3:43815
I20260807 18:13:49.546489 687 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.154.3:43815 every 8 connection(s)
I20260807 18:13:49.549243 505 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126328808105-19048-0/minicluster-data/ts-2/data/info.pb
I20260807 18:13:49.555892 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 505
I20260807 18:13:49.556463 19048 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786126328808105-19048-0/minicluster-data/ts-2/wal/instance
I20260807 18:13:49.558019 19048 external_mini_cluster.cc:1428] Setting key 9d7117d71a615752f21dc1b65733acdc
I20260807 18:13:49.582649 690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:49.554781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.3:43387 (local address 127.18.154.62:39167)
0807 18:13:49.556324 (+ 1543us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:49.556354 (+ 30us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:49.559624 (+ 3270us) server_negotiation.cc:407] Connection header received
0807 18:13:49.565927 (+ 6303us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:49.565935 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:49.566113 (+ 178us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:49.566477 (+ 364us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:49.569349 (+ 2872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:49.571083 (+ 1734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:49.575370 (+ 4287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:49.576343 (+ 973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:49.576637 (+ 294us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:49.580133 (+ 3496us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:13:49.580140 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:13:49.580144 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:13:49.580176 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:13:49.580249 (+ 73us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:13:49.580357 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:13:49.580736 (+ 379us) server_negotiation.cc:1035] Waiting for connection context
0807 18:13:49.581236 (+ 500us) server_negotiation.cc:299] Negotiation successful
0807 18:13:49.581426 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1117,"thread_start_us":641,"threads_started":1}
I20260807 18:13:49.584980 691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:49.556466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:39167 (local address 127.18.154.3:43387)
0807 18:13:49.559314 (+ 2848us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:49.559408 (+ 94us) client_negotiation.cc:174] Beginning negotiation
0807 18:13:49.565216 (+ 5808us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:13:49.567045 (+ 1829us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:13:49.567055 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:13:49.567231 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:13:49.568971 (+ 1740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:49.569056 (+ 85us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:49.571487 (+ 2431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:49.571497 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:49.574942 (+ 3445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:49.574958 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:49.576753 (+ 1795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:49.576773 (+ 20us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:49.577506 (+ 733us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:49.579365 (+ 1859us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 18:13:49.579372 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:13:49.579504 (+ 132us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 18:13:49.579746 (+ 242us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:13:49.580728 (+ 982us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:13:49.580756 (+ 28us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:13:49.580769 (+ 13us) client_negotiation.cc:769] Sending connection context
0807 18:13:49.581189 (+ 420us) client_negotiation.cc:240] Negotiation successful
0807 18:13:49.582293 (+ 1104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1172,"thread_start_us":537,"threads_started":1}
I20260807 18:13:49.588367 688 heartbeater.cc:347] Connected to a master server at 127.18.154.62:39167
I20260807 18:13:49.588861 688 heartbeater.cc:464] Registering TS with master...
I20260807 18:13:49.590791 688 heartbeater.cc:511] Master 127.18.154.62:39167 requested a full tablet report, sending...
I20260807 18:13:49.594897 32488 ts_manager.cc:194] Registered new tserver with Master: dc532e1045a54efc9e2732280656fba0 (127.18.154.3:43815)
I20260807 18:13:49.596735 32488 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.154.3:43387
I20260807 18:13:49.609774 19048 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260807 18:13:49.622496 690 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:13:49.618948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34860 (local address 127.18.154.62:39167)
0807 18:13:49.619425 (+ 477us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:49.619448 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:49.620228 (+ 780us) server_negotiation.cc:407] Connection header received
0807 18:13:49.620710 (+ 482us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:49.620715 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:49.620815 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:49.621005 (+ 190us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:49.621897 (+ 892us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34860: BlockingRecv error: recv got EOF from 127.0.0.1:34860 (error 108)
Metrics: {"server-negotiator.queue_time_us":224}
I20260807 18:13:49.623420 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 32586
I20260807 18:13:49.645731 702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:49.625314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48112 (local address 127.18.154.1:39595)
0807 18:13:49.630392 (+ 5078us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:49.630468 (+ 76us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:49.630545 (+ 77us) server_negotiation.cc:407] Connection header received
0807 18:13:49.630669 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:49.630678 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:49.631136 (+ 458us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:49.631564 (+ 428us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:49.632595 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:49.634933 (+ 2338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:49.637900 (+ 2967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:49.639275 (+ 1375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:49.639597 (+ 322us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:49.642234 (+ 2637us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:13:49.642257 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:13:49.642287 (+ 30us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:13:49.643589 (+ 1302us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:13:49.643694 (+ 105us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:13:49.643733 (+ 39us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:13:49.644093 (+ 360us) server_negotiation.cc:1035] Waiting for connection context
0807 18:13:49.644393 (+ 300us) server_negotiation.cc:299] Negotiation successful
0807 18:13:49.644775 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4763,"thread_start_us":4532,"threads_started":1}
I20260807 18:13:49.648355 302 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:13:49.762979 502 heartbeater.cc:503] Master 127.18.154.62:39167 was elected leader, sending a full tablet report...
I20260807 18:13:49.803788 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 32586
I20260807 18:13:49.851243 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 312
I20260807 18:13:49.868463 704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:49.853234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36878 (local address 127.18.154.2:34621)
0807 18:13:49.854497 (+ 1263us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:49.854512 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:49.854574 (+ 62us) server_negotiation.cc:407] Connection header received
0807 18:13:49.854701 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:49.854719 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:49.855088 (+ 369us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:49.855363 (+ 275us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:49.856023 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:49.858720 (+ 2697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:49.860961 (+ 2241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:49.862486 (+ 1525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:49.862908 (+ 422us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:49.865054 (+ 2146us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:13:49.865072 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:13:49.865089 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:13:49.866409 (+ 1320us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:13:49.866658 (+ 249us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:13:49.866710 (+ 52us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:13:49.866857 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0807 18:13:49.867048 (+ 191us) server_negotiation.cc:299] Negotiation successful
0807 18:13:49.867635 (+ 587us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":949,"thread_start_us":666,"threads_started":1}
I20260807 18:13:49.871901 497 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:13:50.029896 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 312
I20260807 18:13:50.077863 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 505
I20260807 18:13:50.096529 707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:50.080129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47758 (local address 127.18.154.3:43815)
0807 18:13:50.081337 (+ 1208us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:50.081387 (+ 50us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:50.081491 (+ 104us) server_negotiation.cc:407] Connection header received
0807 18:13:50.081798 (+ 307us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:50.081807 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:50.082227 (+ 420us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:50.082863 (+ 636us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:50.083912 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:50.085991 (+ 2079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:50.088535 (+ 2544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:50.090203 (+ 1668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:50.090413 (+ 210us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:50.093013 (+ 2600us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:13:50.093024 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:13:50.093042 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:13:50.094472 (+ 1430us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:13:50.094563 (+ 91us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:13:50.094598 (+ 35us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:13:50.094959 (+ 361us) server_negotiation.cc:1035] Waiting for connection context
0807 18:13:50.095299 (+ 340us) server_negotiation.cc:299] Negotiation successful
0807 18:13:50.095611 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":789,"thread_start_us":478,"threads_started":1}
I20260807 18:13:50.099486 682 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:13:50.247936 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 505
I20260807 18:13:50.300685 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 32445
I20260807 18:13:50.302860 32541 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:13:50.491331 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 32445
2026-08-07T18:13:50Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (3449 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-07T18:13:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:13:50Z Disabled control of system clock
I20260807 18:13:50.657092 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126328808105-19048-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126328808105-19048-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126328808105-19048-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.154.62:35565
--webserver_interface=127.18.154.62
--webserver_port=0
--builtin_ntp_servers=127.18.154.20:46351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.154.62:35565
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260807 18:13:51.512775 715 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:13:51.513582 715 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:13:51.513948 715 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:13:51.533164 715 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:13:51.533469 715 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:13:51.533696 715 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:13:51.533903 715 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:13:51.534020 715 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:13:51.557997 715 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.154.20:46351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126328808105-19048-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126328808105-19048-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.154.62:35565
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.154.62:35565
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.154.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126328808105-19048-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:13:51.561322 715 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:13:51.565856 715 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:13:51.591588 721 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:13:51.591784 723 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:13:51.591934 726 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:13:51.594718 715 server_base.cc:1034] running on GCE node
I20260807 18:13:51.596613 715 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:13:51.599682 715 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:13:51.601341 715 hybrid_clock.cc:648] HybridClock initialized: now 1786126431601208 us; error 103 us; skew 500 ppm
I20260807 18:13:51.602133 715 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 18:13:51.606611 715 webserver.cc:460] Webserver started at http://127.18.154.62:37325/ using document root <none> and password file <none>
I20260807 18:13:51.608105 715 fs_manager.cc:359] Metadata directory not provided
I20260807 18:13:51.608273 715 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:13:51.608819 715 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:13:51.616370 715 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126328808105-19048-0/minicluster-data/master-0/data/instance:
uuid: "c19726f08d494aaba8cbba94cdafb94e"
format_stamp: "Formatted at 2026-08-07 18:13:51 on dist-test-slave-gkw7"
server_key: "008038439b3257a0fd60d5edc36b53cc"
server_key_iv: "d9da317bfa646498278b5952d11e4d34"
server_key_version: "encryptionkey@0"
I20260807 18:13:51.618064 715 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126328808105-19048-0/minicluster-data/master-0/wal/instance:
uuid: "c19726f08d494aaba8cbba94cdafb94e"
format_stamp: "Formatted at 2026-08-07 18:13:51 on dist-test-slave-gkw7"
server_key: "008038439b3257a0fd60d5edc36b53cc"
server_key_iv: "d9da317bfa646498278b5952d11e4d34"
server_key_version: "encryptionkey@0"
I20260807 18:13:51.629943 715 fs_manager.cc:691] Time spent creating directory manager: real 0.011s user 0.008s sys 0.003s
I20260807 18:13:51.639451 735 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:13:51.642750 715 fs_manager.cc:725] Time spent opening block manager: real 0.008s user 0.006s sys 0.000s
I20260807 18:13:51.643251 715 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126328808105-19048-0/minicluster-data/master-0/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126328808105-19048-0/minicluster-data/master-0/wal
uuid: "c19726f08d494aaba8cbba94cdafb94e"
format_stamp: "Formatted at 2026-08-07 18:13:51 on dist-test-slave-gkw7"
server_key: "008038439b3257a0fd60d5edc36b53cc"
server_key_iv: "d9da317bfa646498278b5952d11e4d34"
server_key_version: "encryptionkey@0"
I20260807 18:13:51.643772 715 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126328808105-19048-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126328808105-19048-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126328808105-19048-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:13:51.657006 715 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:13:51.658880 715 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:13:51.659577 715 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:13:51.701828 715 rpc_server.cc:307] RPC server started. Bound to: 127.18.154.62:35565
I20260807 18:13:51.701853 808 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.154.62:35565 every 8 connection(s)
I20260807 18:13:51.704926 715 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
I20260807 18:13:51.710340 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 715
I20260807 18:13:51.710865 19048 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126328808105-19048-0/minicluster-data/master-0/wal/instance
I20260807 18:13:51.713038 19048 external_mini_cluster.cc:1428] Setting key 2aaa1269b1187d8ad74affc7e94179e6
I20260807 18:13:51.716296 810 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:13:51.731065 814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:51.718121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57734 (local address 127.18.154.62:35565)
0807 18:13:51.720384 (+ 2263us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:51.720410 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:51.720494 (+ 84us) server_negotiation.cc:407] Connection header received
0807 18:13:51.723406 (+ 2912us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:51.723448 (+ 42us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:51.724448 (+ 1000us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:51.725364 (+ 916us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:51.727558 (+ 2194us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:13:51.727567 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:13:51.727591 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:13:51.727648 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:13:51.727803 (+ 155us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:13:51.727836 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:13:51.728390 (+ 554us) server_negotiation.cc:1035] Waiting for connection context
0807 18:13:51.728668 (+ 278us) server_negotiation.cc:299] Negotiation successful
0807 18:13:51.729098 (+ 430us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1329,"thread_start_us":942,"threads_started":1}
I20260807 18:13:51.743059 810 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c19726f08d494aaba8cbba94cdafb94e: Bootstrap starting.
I20260807 18:13:51.750032 810 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c19726f08d494aaba8cbba94cdafb94e: Neither blocks nor log segments found. Creating new log.
I20260807 18:13:51.752404 810 log.cc:824] T 00000000000000000000000000000000 P c19726f08d494aaba8cbba94cdafb94e: Log is configured to *not* fsync() on all Append() calls
I20260807 18:13:51.758562 810 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c19726f08d494aaba8cbba94cdafb94e: No bootstrap required, opened a new log
I20260807 18:13:51.773386 810 raft_consensus.cc:348] T 00000000000000000000000000000000 P c19726f08d494aaba8cbba94cdafb94e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c19726f08d494aaba8cbba94cdafb94e" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 35565 } }
I20260807 18:13:51.774463 810 raft_consensus.cc:374] T 00000000000000000000000000000000 P c19726f08d494aaba8cbba94cdafb94e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:13:51.774685 810 raft_consensus.cc:729] T 00000000000000000000000000000000 P c19726f08d494aaba8cbba94cdafb94e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c19726f08d494aaba8cbba94cdafb94e, State: Initialized, Role: FOLLOWER
I20260807 18:13:51.775923 810 consensus_queue.cc:260] T 00000000000000000000000000000000 P c19726f08d494aaba8cbba94cdafb94e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c19726f08d494aaba8cbba94cdafb94e" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 35565 } }
I20260807 18:13:51.776376 810 raft_consensus.cc:388] T 00000000000000000000000000000000 P c19726f08d494aaba8cbba94cdafb94e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:13:51.776638 810 raft_consensus.cc:482] T 00000000000000000000000000000000 P c19726f08d494aaba8cbba94cdafb94e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:13:51.777102 810 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c19726f08d494aaba8cbba94cdafb94e [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:13:51.783234 810 raft_consensus.cc:504] T 00000000000000000000000000000000 P c19726f08d494aaba8cbba94cdafb94e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c19726f08d494aaba8cbba94cdafb94e" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 35565 } }
I20260807 18:13:51.784533 810 leader_election.cc:302] T 00000000000000000000000000000000 P c19726f08d494aaba8cbba94cdafb94e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c19726f08d494aaba8cbba94cdafb94e; no voters:
I20260807 18:13:51.785761 810 leader_election.cc:288] T 00000000000000000000000000000000 P c19726f08d494aaba8cbba94cdafb94e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:13:51.786211 816 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c19726f08d494aaba8cbba94cdafb94e [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:13:51.787755 816 raft_consensus.cc:686] T 00000000000000000000000000000000 P c19726f08d494aaba8cbba94cdafb94e [term 1 LEADER]: Becoming Leader. State: Replica: c19726f08d494aaba8cbba94cdafb94e, State: Running, Role: LEADER
I20260807 18:13:51.788923 816 consensus_queue.cc:237] T 00000000000000000000000000000000 P c19726f08d494aaba8cbba94cdafb94e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c19726f08d494aaba8cbba94cdafb94e" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 35565 } }
I20260807 18:13:51.790530 810 sys_catalog.cc:564] T 00000000000000000000000000000000 P c19726f08d494aaba8cbba94cdafb94e [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:13:51.794277 819 sys_catalog.cc:455] T 00000000000000000000000000000000 P c19726f08d494aaba8cbba94cdafb94e [sys.catalog]: SysCatalogTable state changed. Reason: New leader c19726f08d494aaba8cbba94cdafb94e. Latest consensus state: current_term: 1 leader_uuid: "c19726f08d494aaba8cbba94cdafb94e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c19726f08d494aaba8cbba94cdafb94e" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 35565 } } }
I20260807 18:13:51.795054 819 sys_catalog.cc:458] T 00000000000000000000000000000000 P c19726f08d494aaba8cbba94cdafb94e [sys.catalog]: This master's current role is: LEADER
I20260807 18:13:51.795969 817 sys_catalog.cc:455] T 00000000000000000000000000000000 P c19726f08d494aaba8cbba94cdafb94e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c19726f08d494aaba8cbba94cdafb94e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c19726f08d494aaba8cbba94cdafb94e" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 35565 } } }
I20260807 18:13:51.796909 817 sys_catalog.cc:458] T 00000000000000000000000000000000 P c19726f08d494aaba8cbba94cdafb94e [sys.catalog]: This master's current role is: LEADER
I20260807 18:13:51.801724 826 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:13:51.812677 826 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:13:51.830009 826 catalog_manager.cc:1347] Generated new cluster ID: 2a85d052be6a40caa57937dfa9f7ce73
I20260807 18:13:51.830252 826 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:13:51.852717 826 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:13:51.853925 826 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:13:51.865397 826 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c19726f08d494aaba8cbba94cdafb94e: Generated new TSK 0
I20260807 18:13:51.867898 826 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:13:51.878844 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126328808105-19048-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126328808105-19048-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126328808105-19048-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126328808105-19048-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.154.1:0
--local_ip_for_outbound_sockets=127.18.154.1
--webserver_interface=127.18.154.1
--webserver_port=0
--tserver_master_addrs=127.18.154.62:35565
--builtin_ntp_servers=127.18.154.20:46351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260807 18:13:52.255558 846 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:13:52.255867 846 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:13:52.255960 846 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:13:52.263399 846 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:13:52.263511 846 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:13:52.263626 846 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.154.1
I20260807 18:13:52.273252 846 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.154.20:46351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126328808105-19048-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126328808105-19048-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.154.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126328808105-19048-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.154.1
--webserver_port=0
--tserver_master_addrs=127.18.154.62:35565
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.154.1
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126328808105-19048-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:13:52.274619 846 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:13:52.276386 846 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:13:52.290418 851 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:13:52.290516 853 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:13:52.290720 855 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:13:52.291679 846 server_base.cc:1034] running on GCE node
I20260807 18:13:52.292757 846 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:13:52.293926 846 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:13:52.295192 846 hybrid_clock.cc:648] HybridClock initialized: now 1786126432295113 us; error 47 us; skew 500 ppm
I20260807 18:13:52.295581 846 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 18:13:52.298969 846 webserver.cc:460] Webserver started at http://127.18.154.1:46015/ using document root <none> and password file <none>
I20260807 18:13:52.299934 846 fs_manager.cc:359] Metadata directory not provided
I20260807 18:13:52.300074 846 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:13:52.300475 846 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:13:52.303666 846 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126328808105-19048-0/minicluster-data/ts-0/data/instance:
uuid: "8b936348ad6744e08a652414799a2e9a"
format_stamp: "Formatted at 2026-08-07 18:13:52 on dist-test-slave-gkw7"
server_key: "50be61fa68d7ba231d77604ad7652d0b"
server_key_iv: "4aeef97f62b96972c966dad2d99be7bd"
server_key_version: "encryptionkey@0"
I20260807 18:13:52.304530 846 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126328808105-19048-0/minicluster-data/ts-0/wal/instance:
uuid: "8b936348ad6744e08a652414799a2e9a"
format_stamp: "Formatted at 2026-08-07 18:13:52 on dist-test-slave-gkw7"
server_key: "50be61fa68d7ba231d77604ad7652d0b"
server_key_iv: "4aeef97f62b96972c966dad2d99be7bd"
server_key_version: "encryptionkey@0"
I20260807 18:13:52.310075 846 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260807 18:13:52.313849 866 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:13:52.315374 846 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 18:13:52.315513 846 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126328808105-19048-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126328808105-19048-0/minicluster-data/ts-0/wal
uuid: "8b936348ad6744e08a652414799a2e9a"
format_stamp: "Formatted at 2026-08-07 18:13:52 on dist-test-slave-gkw7"
server_key: "50be61fa68d7ba231d77604ad7652d0b"
server_key_iv: "4aeef97f62b96972c966dad2d99be7bd"
server_key_version: "encryptionkey@0"
I20260807 18:13:52.315703 846 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126328808105-19048-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126328808105-19048-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126328808105-19048-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:13:52.321718 846 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:13:52.322570 846 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:13:52.322805 846 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:13:52.323716 846 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:13:52.325162 846 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:13:52.325280 846 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:13:52.325364 846 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:13:52.325420 846 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:13:52.361310 846 rpc_server.cc:307] RPC server started. Bound to: 127.18.154.1:35013
I20260807 18:13:52.361361 1025 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.154.1:35013 every 8 connection(s)
I20260807 18:13:52.362815 846 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126328808105-19048-0/minicluster-data/ts-0/data/info.pb
I20260807 18:13:52.371754 1030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:52.365794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.1:49965 (local address 127.18.154.62:35565)
0807 18:13:52.366230 (+ 436us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:52.366235 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:52.367559 (+ 1324us) server_negotiation.cc:407] Connection header received
0807 18:13:52.369286 (+ 1727us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:52.369291 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:52.369369 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:52.369494 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:52.370919 (+ 1425us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:13:52.370926 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:13:52.370931 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:13:52.370963 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:13:52.371012 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:13:52.371024 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:13:52.371161 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0807 18:13:52.371359 (+ 198us) server_negotiation.cc:299] Negotiation successful
0807 18:13:52.371431 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":324,"thread_start_us":226,"threads_started":1}
I20260807 18:13:52.372629 1031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:52.366464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:35565 (local address 127.18.154.1:49965)
0807 18:13:52.367402 (+ 938us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:52.367439 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 18:13:52.369010 (+ 1571us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:13:52.369757 (+ 747us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:13:52.369766 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:13:52.369858 (+ 92us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:13:52.370582 (+ 724us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 18:13:52.370584 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:13:52.370623 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 18:13:52.370719 (+ 96us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:13:52.371125 (+ 406us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:13:52.371139 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:13:52.371147 (+ 8us) client_negotiation.cc:769] Sending connection context
0807 18:13:52.371442 (+ 295us) client_negotiation.cc:240] Negotiation successful
0807 18:13:52.371694 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":271,"threads_started":1}
I20260807 18:13:52.372648 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 846
I20260807 18:13:52.372879 19048 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126328808105-19048-0/minicluster-data/ts-0/wal/instance
I20260807 18:13:52.373963 19048 external_mini_cluster.cc:1428] Setting key 7a944bd042fd9009375d4a60fd4f0721
I20260807 18:13:52.374441 1027 heartbeater.cc:347] Connected to a master server at 127.18.154.62:35565
I20260807 18:13:52.374804 1027 heartbeater.cc:464] Registering TS with master...
I20260807 18:13:52.375449 1027 heartbeater.cc:511] Master 127.18.154.62:35565 requested a full tablet report, sending...
I20260807 18:13:52.377015 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126328808105-19048-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126328808105-19048-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126328808105-19048-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126328808105-19048-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.154.2:0
--local_ip_for_outbound_sockets=127.18.154.2
--webserver_interface=127.18.154.2
--webserver_port=0
--tserver_master_addrs=127.18.154.62:35565
--builtin_ntp_servers=127.18.154.20:46351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260807 18:13:52.377784 762 ts_manager.cc:194] Registered new tserver with Master: 8b936348ad6744e08a652414799a2e9a (127.18.154.1:35013)
W20260807 18:13:52.748274 1032 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:13:52.748577 1032 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:13:52.748718 1032 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:13:52.756342 1032 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:13:52.756453 1032 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:13:52.756639 1032 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.154.2
I20260807 18:13:52.765846 1032 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.154.20:46351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126328808105-19048-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126328808105-19048-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.154.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126328808105-19048-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.154.2
--webserver_port=0
--tserver_master_addrs=127.18.154.62:35565
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.154.2
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126328808105-19048-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:13:52.767278 1032 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:13:52.769102 1032 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:13:52.782907 1045 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:13:52.782912 1041 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:13:52.783179 1040 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:13:52.783784 1032 server_base.cc:1034] running on GCE node
I20260807 18:13:52.784507 1032 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:13:52.785516 1032 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:13:52.786795 1032 hybrid_clock.cc:648] HybridClock initialized: now 1786126432786745 us; error 42 us; skew 500 ppm
I20260807 18:13:52.787133 1032 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 18:13:52.789492 1032 webserver.cc:460] Webserver started at http://127.18.154.2:46301/ using document root <none> and password file <none>
I20260807 18:13:52.790320 1032 fs_manager.cc:359] Metadata directory not provided
I20260807 18:13:52.790443 1032 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:13:52.790824 1032 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:13:52.794051 1032 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126328808105-19048-0/minicluster-data/ts-1/data/instance:
uuid: "98b1701304eb4f9bb6bbd541531a719e"
format_stamp: "Formatted at 2026-08-07 18:13:52 on dist-test-slave-gkw7"
server_key: "71077a52498e319a29a3efd438745eb6"
server_key_iv: "1cd6731217868e83adfd92be4dbf3379"
server_key_version: "encryptionkey@0"
I20260807 18:13:52.794903 1032 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126328808105-19048-0/minicluster-data/ts-1/wal/instance:
uuid: "98b1701304eb4f9bb6bbd541531a719e"
format_stamp: "Formatted at 2026-08-07 18:13:52 on dist-test-slave-gkw7"
server_key: "71077a52498e319a29a3efd438745eb6"
server_key_iv: "1cd6731217868e83adfd92be4dbf3379"
server_key_version: "encryptionkey@0"
I20260807 18:13:52.800401 1032 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 18:13:52.804322 1053 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:13:52.805908 1032 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:13:52.806092 1032 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126328808105-19048-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126328808105-19048-0/minicluster-data/ts-1/wal
uuid: "98b1701304eb4f9bb6bbd541531a719e"
format_stamp: "Formatted at 2026-08-07 18:13:52 on dist-test-slave-gkw7"
server_key: "71077a52498e319a29a3efd438745eb6"
server_key_iv: "1cd6731217868e83adfd92be4dbf3379"
server_key_version: "encryptionkey@0"
I20260807 18:13:52.806326 1032 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126328808105-19048-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126328808105-19048-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126328808105-19048-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:13:52.812592 1032 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:13:52.813475 1032 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:13:52.813827 1032 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:13:52.814847 1032 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:13:52.816355 1032 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:13:52.816444 1032 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:13:52.816519 1032 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:13:52.816574 1032 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:13:52.852119 1032 rpc_server.cc:307] RPC server started. Bound to: 127.18.154.2:39761
I20260807 18:13:52.852223 1220 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.154.2:39761 every 8 connection(s)
I20260807 18:13:52.853657 1032 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126328808105-19048-0/minicluster-data/ts-1/data/info.pb
I20260807 18:13:52.858139 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 1032
I20260807 18:13:52.858317 19048 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126328808105-19048-0/minicluster-data/ts-1/wal/instance
I20260807 18:13:52.858814 19048 external_mini_cluster.cc:1428] Setting key 5b2d507863a41bb00389c5fe125e749c
I20260807 18:13:52.861954 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126328808105-19048-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126328808105-19048-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126328808105-19048-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126328808105-19048-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.154.3:0
--local_ip_for_outbound_sockets=127.18.154.3
--webserver_interface=127.18.154.3
--webserver_port=0
--tserver_master_addrs=127.18.154.62:35565
--builtin_ntp_servers=127.18.154.20:46351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260807 18:13:52.862775 1030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:52.856501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.2:56347 (local address 127.18.154.62:35565)
0807 18:13:52.856736 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:52.856743 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:52.858444 (+ 1701us) server_negotiation.cc:407] Connection header received
0807 18:13:52.860356 (+ 1912us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:52.860361 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:52.860429 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:52.860560 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:52.862011 (+ 1451us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:13:52.862018 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:13:52.862024 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:13:52.862046 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:13:52.862096 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:13:52.862116 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:13:52.862253 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0807 18:13:52.862448 (+ 195us) server_negotiation.cc:299] Negotiation successful
0807 18:13:52.862520 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260807 18:13:52.863873 1226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:52.857122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:35565 (local address 127.18.154.2:56347)
0807 18:13:52.858182 (+ 1060us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:52.858251 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0807 18:13:52.860079 (+ 1828us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:13:52.860810 (+ 731us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:13:52.860818 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:13:52.860917 (+ 99us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:13:52.861693 (+ 776us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 18:13:52.861696 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:13:52.861737 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 18:13:52.861839 (+ 102us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:13:52.862211 (+ 372us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:13:52.862223 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:13:52.862233 (+ 10us) client_negotiation.cc:769] Sending connection context
0807 18:13:52.862536 (+ 303us) client_negotiation.cc:240] Negotiation successful
0807 18:13:52.862826 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":571,"thread_start_us":346,"threads_started":1}
I20260807 18:13:52.865408 1221 heartbeater.cc:347] Connected to a master server at 127.18.154.62:35565
I20260807 18:13:52.865754 1221 heartbeater.cc:464] Registering TS with master...
I20260807 18:13:52.866349 1221 heartbeater.cc:511] Master 127.18.154.62:35565 requested a full tablet report, sending...
I20260807 18:13:52.867808 762 ts_manager.cc:194] Registered new tserver with Master: 98b1701304eb4f9bb6bbd541531a719e (127.18.154.2:39761)
W20260807 18:13:53.231000 1227 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:13:53.231289 1227 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:13:53.231463 1227 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:13:53.238920 1227 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:13:53.239025 1227 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:13:53.239231 1227 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.154.3
I20260807 18:13:53.249096 1227 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.154.20:46351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126328808105-19048-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126328808105-19048-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.154.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126328808105-19048-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.154.3
--webserver_port=0
--tserver_master_addrs=127.18.154.62:35565
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.154.3
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126328808105-19048-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:13:53.250537 1227 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:13:53.252424 1227 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:13:53.266266 1233 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:13:53.266377 1240 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:13:53.266310 1236 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:13:53.267472 1227 server_base.cc:1034] running on GCE node
I20260807 18:13:53.268219 1227 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:13:53.269263 1227 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:13:53.270490 1227 hybrid_clock.cc:648] HybridClock initialized: now 1786126433270422 us; error 85 us; skew 500 ppm
I20260807 18:13:53.270841 1227 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 18:13:53.273761 1227 webserver.cc:460] Webserver started at http://127.18.154.3:42583/ using document root <none> and password file <none>
I20260807 18:13:53.274741 1227 fs_manager.cc:359] Metadata directory not provided
I20260807 18:13:53.274854 1227 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:13:53.275247 1227 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:13:53.279467 1227 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126328808105-19048-0/minicluster-data/ts-2/data/instance:
uuid: "5cd3d148dcdf4a15a825541d8c69defb"
format_stamp: "Formatted at 2026-08-07 18:13:53 on dist-test-slave-gkw7"
server_key: "088bb99de1f24391c20cf5d21fb398b7"
server_key_iv: "7699d4205fe42b2463b24564139d4aa0"
server_key_version: "encryptionkey@0"
I20260807 18:13:53.280437 1227 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126328808105-19048-0/minicluster-data/ts-2/wal/instance:
uuid: "5cd3d148dcdf4a15a825541d8c69defb"
format_stamp: "Formatted at 2026-08-07 18:13:53 on dist-test-slave-gkw7"
server_key: "088bb99de1f24391c20cf5d21fb398b7"
server_key_iv: "7699d4205fe42b2463b24564139d4aa0"
server_key_version: "encryptionkey@0"
I20260807 18:13:53.287022 1227 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.004s
I20260807 18:13:53.290392 1250 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:13:53.291818 1227 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 18:13:53.291980 1227 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126328808105-19048-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126328808105-19048-0/minicluster-data/ts-2/wal
uuid: "5cd3d148dcdf4a15a825541d8c69defb"
format_stamp: "Formatted at 2026-08-07 18:13:53 on dist-test-slave-gkw7"
server_key: "088bb99de1f24391c20cf5d21fb398b7"
server_key_iv: "7699d4205fe42b2463b24564139d4aa0"
server_key_version: "encryptionkey@0"
I20260807 18:13:53.292228 1227 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126328808105-19048-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126328808105-19048-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126328808105-19048-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:13:53.298218 1227 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:13:53.298981 1227 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:13:53.299224 1227 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:13:53.300426 1227 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:13:53.302271 1227 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:13:53.302393 1227 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:13:53.302510 1227 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:13:53.302598 1227 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:13:53.338737 1227 rpc_server.cc:307] RPC server started. Bound to: 127.18.154.3:40873
I20260807 18:13:53.338832 1418 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.154.3:40873 every 8 connection(s)
I20260807 18:13:53.340155 1227 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126328808105-19048-0/minicluster-data/ts-2/data/info.pb
I20260807 18:13:53.342958 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 1227
I20260807 18:13:53.343181 19048 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786126328808105-19048-0/minicluster-data/ts-2/wal/instance
I20260807 18:13:53.343638 19048 external_mini_cluster.cc:1428] Setting key 22a193b7cbd869bbe826dff83599b29d
I20260807 18:13:53.349125 1030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:53.342977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.3:58841 (local address 127.18.154.62:35565)
0807 18:13:53.343228 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:53.343236 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:53.344890 (+ 1654us) server_negotiation.cc:407] Connection header received
0807 18:13:53.347079 (+ 2189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:53.347083 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:53.347135 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:53.347217 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:53.348492 (+ 1275us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:13:53.348497 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:13:53.348500 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:13:53.348515 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:13:53.348548 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:13:53.348559 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:13:53.348648 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0807 18:13:53.348842 (+ 194us) server_negotiation.cc:299] Negotiation successful
0807 18:13:53.348893 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260807 18:13:53.350159 1421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:53.343515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:35565 (local address 127.18.154.3:58841)
0807 18:13:53.344706 (+ 1191us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:53.344759 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0807 18:13:53.346768 (+ 2009us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:13:53.347445 (+ 677us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:13:53.347453 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:13:53.347547 (+ 94us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:13:53.348205 (+ 658us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 18:13:53.348208 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:13:53.348246 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 18:13:53.348334 (+ 88us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:13:53.348661 (+ 327us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:13:53.348669 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:13:53.348674 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 18:13:53.348893 (+ 219us) client_negotiation.cc:240] Negotiation successful
0807 18:13:53.349093 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":598,"thread_start_us":346,"threads_started":1}
I20260807 18:13:53.351522 1419 heartbeater.cc:347] Connected to a master server at 127.18.154.62:35565
I20260807 18:13:53.351859 1419 heartbeater.cc:464] Registering TS with master...
I20260807 18:13:53.352488 1419 heartbeater.cc:511] Master 127.18.154.62:35565 requested a full tablet report, sending...
I20260807 18:13:53.354027 762 ts_manager.cc:194] Registered new tserver with Master: 5cd3d148dcdf4a15a825541d8c69defb (127.18.154.3:40873)
I20260807 18:13:53.359400 19048 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260807 18:13:53.366496 1030 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:13:53.364720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57748 (local address 127.18.154.62:35565)
0807 18:13:53.364926 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:53.364933 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:53.365525 (+ 592us) server_negotiation.cc:407] Connection header received
0807 18:13:53.365744 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:53.365750 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:53.365813 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:53.365917 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:53.366283 (+ 366us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57748: BlockingRecv error: recv got EOF from 127.0.0.1:57748 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
I20260807 18:13:53.367812 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 846
I20260807 18:13:53.371469 1432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:53.368894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43826 (local address 127.18.154.1:35013)
0807 18:13:53.369469 (+ 575us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:53.369477 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:53.369501 (+ 24us) server_negotiation.cc:407] Connection header received
0807 18:13:53.369609 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:53.369615 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:53.369805 (+ 190us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:53.369929 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:53.370871 (+ 942us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:13:53.370877 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:13:53.370887 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:13:53.370899 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:13:53.370940 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:13:53.370953 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:13:53.371080 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0807 18:13:53.371132 (+ 52us) server_negotiation.cc:299] Negotiation successful
0807 18:13:53.371218 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":435,"thread_start_us":302,"threads_started":1}
I20260807 18:13:53.372690 1017 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:13:53.453872 1027 heartbeater.cc:503] Master 127.18.154.62:35565 was elected leader, sending a full tablet report...
I20260807 18:13:53.455346 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 846
W20260807 18:13:53.478762 741 connection.cc:535] server connection from 127.18.154.1:49965 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260807 18:13:53.479156 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 1032
I20260807 18:13:53.482738 1437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:53.480085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45342 (local address 127.18.154.2:39761)
0807 18:13:53.480657 (+ 572us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:53.480665 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:53.480693 (+ 28us) server_negotiation.cc:407] Connection header received
0807 18:13:53.480758 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:53.480764 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:53.480986 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:53.481127 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:53.482094 (+ 967us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:13:53.482100 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:13:53.482112 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:13:53.482129 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:13:53.482177 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:13:53.482191 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:13:53.482298 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0807 18:13:53.482418 (+ 120us) server_negotiation.cc:299] Negotiation successful
0807 18:13:53.482503 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":407,"thread_start_us":283,"threads_started":1}
I20260807 18:13:53.483997 1216 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:13:53.566056 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 1032
I20260807 18:13:53.587949 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 1227
I20260807 18:13:53.592518 1440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:53.589253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34250 (local address 127.18.154.3:40873)
0807 18:13:53.590114 (+ 861us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:53.590121 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:53.590156 (+ 35us) server_negotiation.cc:407] Connection header received
0807 18:13:53.590220 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:53.590227 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:53.590461 (+ 234us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:53.590594 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:53.591686 (+ 1092us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:13:53.591693 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:13:53.591705 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:13:53.591722 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:13:53.591788 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:13:53.591810 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:13:53.592000 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0807 18:13:53.592071 (+ 71us) server_negotiation.cc:299] Negotiation successful
0807 18:13:53.592157 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":702,"thread_start_us":506,"threads_started":1}
I20260807 18:13:53.594470 1413 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:13:53.676546 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 1227
I20260807 18:13:53.699374 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 715
I20260807 18:13:53.700984 803 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:13:53.805985 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 715
2026-08-07T18:13:53Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (3287 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126328808105-19048-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:13:53 dist-test-slave-gkw7 krb5kdc[1446](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:13:53 dist-test-slave-gkw7 krb5kdc[1446](info): set up 2 sockets
Aug 07 18:13:53 dist-test-slave-gkw7 krb5kdc[1446](info): commencing operation
krb5kdc: starting...
W20260807 18:13:55.181412 19048 mini_kdc.cc:121] Time spent starting KDC: real 1.331s user 0.003s sys 0.038s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126328808105-19048-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126328808105-19048-0/krb5kdc/test-user.keytab.
Aug 07 18:13:55 dist-test-slave-gkw7 krb5kdc[1446](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126435, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:13:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:13:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.154.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.154.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.154.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.62.keytab.
Entry for principal kudu/127.18.154.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.62.keytab.
I20260807 18:13:55.477083 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126328808105-19048-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126328808105-19048-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126328808105-19048-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.154.62:37603
--webserver_interface=127.18.154.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.62.keytab
--principal=kudu/127.18.154.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.154.20:34025
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.154.62:37603
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126328808105-19048-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126328808105-19048-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126328808105-19048-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:13:55.824960 1464 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:13:55.825306 1464 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:13:55.825475 1464 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:13:55.833168 1464 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:13:55.833307 1464 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:13:55.833391 1464 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:13:55.833493 1464 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:13:55.833576 1464 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:13:55.844233 1464 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.154.20:34025
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126328808105-19048-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126328808105-19048-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.154.62:37603
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.62.keytab
--principal=kudu/127.18.154.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.154.62:37603
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.154.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126328808105-19048-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:13:55.845767 1464 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:13:55.847162 1464 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:13:55.859709 1472 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:13:55.859794 1477 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:13:55.859753 1474 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:13:55.860870 1464 server_base.cc:1034] running on GCE node
I20260807 18:13:55.861757 1464 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:13:55.863173 1464 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:13:55.864336 1464 hybrid_clock.cc:648] HybridClock initialized: now 1786126435864269 us; error 50 us; skew 500 ppm
I20260807 18:13:55.864718 1464 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:13:55 dist-test-slave-gkw7 krb5kdc[1446](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126435, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:13:55.868121 1464 init.cc:377] Logged in from keytab as kudu/127.18.154.62@KRBTEST.COM (short username kudu)
I20260807 18:13:55.869678 1464 webserver.cc:460] Webserver started at http://127.18.154.62:43845/ using document root <none> and password file <none>
I20260807 18:13:55.870388 1464 fs_manager.cc:359] Metadata directory not provided
I20260807 18:13:55.870527 1464 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:13:55.870877 1464 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:13:55.874042 1464 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126328808105-19048-0/minicluster-data/master-0/data/instance:
uuid: "a5fd322e4dd64f8eb8309524ebd988c2"
format_stamp: "Formatted at 2026-08-07 18:13:55 on dist-test-slave-gkw7"
server_key: "e233391206274cbda7289f47a6ab46c7"
server_key_iv: "ad963bba33f3f75e61ba1e8398cfa942"
server_key_version: "encryptionkey@0"
I20260807 18:13:55.874833 1464 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126328808105-19048-0/minicluster-data/master-0/wal/instance:
uuid: "a5fd322e4dd64f8eb8309524ebd988c2"
format_stamp: "Formatted at 2026-08-07 18:13:55 on dist-test-slave-gkw7"
server_key: "e233391206274cbda7289f47a6ab46c7"
server_key_iv: "ad963bba33f3f75e61ba1e8398cfa942"
server_key_version: "encryptionkey@0"
I20260807 18:13:55.880088 1464 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 18:13:55.883625 1485 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:13:55.884980 1464 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 18:13:55.885172 1464 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126328808105-19048-0/minicluster-data/master-0/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126328808105-19048-0/minicluster-data/master-0/wal
uuid: "a5fd322e4dd64f8eb8309524ebd988c2"
format_stamp: "Formatted at 2026-08-07 18:13:55 on dist-test-slave-gkw7"
server_key: "e233391206274cbda7289f47a6ab46c7"
server_key_iv: "ad963bba33f3f75e61ba1e8398cfa942"
server_key_version: "encryptionkey@0"
I20260807 18:13:55.885361 1464 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126328808105-19048-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126328808105-19048-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126328808105-19048-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:13:55.902817 1464 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:13:55.907153 1464 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:13:55.907464 1464 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:13:55.927075 1464 rpc_server.cc:307] RPC server started. Bound to: 127.18.154.62:37603
I20260807 18:13:55.927188 1552 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.154.62:37603 every 8 connection(s)
I20260807 18:13:55.928551 1464 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
I20260807 18:13:55.932462 1553 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:13:55.936328 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 1464
I20260807 18:13:55.936547 19048 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126328808105-19048-0/minicluster-data/master-0/wal/instance
I20260807 18:13:55.937494 19048 external_mini_cluster.cc:1428] Setting key c81913382c0d66978d02b56d8c816ced
I20260807 18:13:55.946827 1553 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a5fd322e4dd64f8eb8309524ebd988c2: Bootstrap starting.
I20260807 18:13:55.950994 1553 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a5fd322e4dd64f8eb8309524ebd988c2: Neither blocks nor log segments found. Creating new log.
I20260807 18:13:55.952486 1553 log.cc:824] T 00000000000000000000000000000000 P a5fd322e4dd64f8eb8309524ebd988c2: Log is configured to *not* fsync() on all Append() calls
Aug 07 18:13:55 dist-test-slave-gkw7 krb5kdc[1446](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126435, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.154.62@KRBTEST.COM
I20260807 18:13:55.955966 1553 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a5fd322e4dd64f8eb8309524ebd988c2: No bootstrap required, opened a new log
I20260807 18:13:55.962981 1556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:55.939843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38550 (local address 127.18.154.62:37603)
0807 18:13:55.940643 (+ 800us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:55.940655 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:55.940698 (+ 43us) server_negotiation.cc:407] Connection header received
0807 18:13:55.942417 (+ 1719us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:55.942440 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:55.942860 (+ 420us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:55.943205 (+ 345us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:55.945223 (+ 2018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:55.946621 (+ 1398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:55.948422 (+ 1801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:55.949202 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:55.949351 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:55.954910 (+ 5559us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:13:55.954919 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:13:55.954936 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:13:55.954976 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:13:55.958321 (+ 3345us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:55.959103 (+ 782us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:55.959114 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:55.959125 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:55.959304 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:55.959715 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:55.959722 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:55.959730 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:55.960282 (+ 552us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:13:55.960640 (+ 358us) server_negotiation.cc:1035] Waiting for connection context
0807 18:13:55.961075 (+ 435us) server_negotiation.cc:299] Negotiation successful
0807 18:13:55.961859 (+ 784us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":588,"thread_start_us":382,"threads_started":1}
I20260807 18:13:55.964949 1553 raft_consensus.cc:348] T 00000000000000000000000000000000 P a5fd322e4dd64f8eb8309524ebd988c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5fd322e4dd64f8eb8309524ebd988c2" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 37603 } }
I20260807 18:13:55.965396 1553 raft_consensus.cc:374] T 00000000000000000000000000000000 P a5fd322e4dd64f8eb8309524ebd988c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:13:55.965591 1553 raft_consensus.cc:729] T 00000000000000000000000000000000 P a5fd322e4dd64f8eb8309524ebd988c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a5fd322e4dd64f8eb8309524ebd988c2, State: Initialized, Role: FOLLOWER
I20260807 18:13:55.966390 1553 consensus_queue.cc:260] T 00000000000000000000000000000000 P a5fd322e4dd64f8eb8309524ebd988c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5fd322e4dd64f8eb8309524ebd988c2" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 37603 } }
I20260807 18:13:55.966677 1553 raft_consensus.cc:388] T 00000000000000000000000000000000 P a5fd322e4dd64f8eb8309524ebd988c2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:13:55.966849 1553 raft_consensus.cc:482] T 00000000000000000000000000000000 P a5fd322e4dd64f8eb8309524ebd988c2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:13:55.967054 1553 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a5fd322e4dd64f8eb8309524ebd988c2 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:13:55.969882 1553 raft_consensus.cc:504] T 00000000000000000000000000000000 P a5fd322e4dd64f8eb8309524ebd988c2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5fd322e4dd64f8eb8309524ebd988c2" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 37603 } }
I20260807 18:13:55.970508 1553 leader_election.cc:302] T 00000000000000000000000000000000 P a5fd322e4dd64f8eb8309524ebd988c2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a5fd322e4dd64f8eb8309524ebd988c2; no voters:
I20260807 18:13:55.971163 1553 leader_election.cc:288] T 00000000000000000000000000000000 P a5fd322e4dd64f8eb8309524ebd988c2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:13:55.971272 1559 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a5fd322e4dd64f8eb8309524ebd988c2 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:13:55.972031 1559 raft_consensus.cc:686] T 00000000000000000000000000000000 P a5fd322e4dd64f8eb8309524ebd988c2 [term 1 LEADER]: Becoming Leader. State: Replica: a5fd322e4dd64f8eb8309524ebd988c2, State: Running, Role: LEADER
I20260807 18:13:55.972669 1559 consensus_queue.cc:237] T 00000000000000000000000000000000 P a5fd322e4dd64f8eb8309524ebd988c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5fd322e4dd64f8eb8309524ebd988c2" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 37603 } }
I20260807 18:13:55.973517 1553 sys_catalog.cc:564] T 00000000000000000000000000000000 P a5fd322e4dd64f8eb8309524ebd988c2 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:13:55.975533 1560 sys_catalog.cc:455] T 00000000000000000000000000000000 P a5fd322e4dd64f8eb8309524ebd988c2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a5fd322e4dd64f8eb8309524ebd988c2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5fd322e4dd64f8eb8309524ebd988c2" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 37603 } } }
I20260807 18:13:55.976039 1560 sys_catalog.cc:458] T 00000000000000000000000000000000 P a5fd322e4dd64f8eb8309524ebd988c2 [sys.catalog]: This master's current role is: LEADER
I20260807 18:13:55.979807 1574 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:13:55.979651 1561 sys_catalog.cc:455] T 00000000000000000000000000000000 P a5fd322e4dd64f8eb8309524ebd988c2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a5fd322e4dd64f8eb8309524ebd988c2. Latest consensus state: current_term: 1 leader_uuid: "a5fd322e4dd64f8eb8309524ebd988c2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5fd322e4dd64f8eb8309524ebd988c2" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 37603 } } }
I20260807 18:13:55.980024 1561 sys_catalog.cc:458] T 00000000000000000000000000000000 P a5fd322e4dd64f8eb8309524ebd988c2 [sys.catalog]: This master's current role is: LEADER
I20260807 18:13:55.985774 1574 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:13:55.996433 1574 catalog_manager.cc:1347] Generated new cluster ID: c4d0862ba1f44473bd320fd060e2c4d5
I20260807 18:13:55.996603 1574 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:13:56.007704 1574 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:13:56.009052 1574 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:13:56.016239 1574 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a5fd322e4dd64f8eb8309524ebd988c2: Generated new TSK 0
I20260807 18:13:56.017109 1574 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.154.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.154.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.154.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.1.keytab.
Entry for principal kudu/127.18.154.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.1.keytab.
I20260807 18:13:56.097711 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126328808105-19048-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126328808105-19048-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126328808105-19048-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126328808105-19048-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.154.1:0
--local_ip_for_outbound_sockets=127.18.154.1
--webserver_interface=127.18.154.1
--webserver_port=0
--tserver_master_addrs=127.18.154.62:37603
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.1.keytab
--principal=kudu/127.18.154.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.154.20:34025
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126328808105-19048-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126328808105-19048-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126328808105-19048-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:13:56.453729 1588 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:13:56.454027 1588 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:13:56.454193 1588 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:13:56.461818 1588 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:13:56.461935 1588 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:13:56.462145 1588 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.154.1
I20260807 18:13:56.472040 1588 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.154.20:34025
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126328808105-19048-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126328808105-19048-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.1.keytab
--principal=kudu/127.18.154.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.154.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126328808105-19048-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.154.1
--webserver_port=0
--tserver_master_addrs=127.18.154.62:37603
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.154.1
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126328808105-19048-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:13:56.473467 1588 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:13:56.475010 1588 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:13:56.488960 1598 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:13:56.489022 1600 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:13:56.489140 1597 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:13:56.489764 1588 server_base.cc:1034] running on GCE node
I20260807 18:13:56.490403 1588 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:13:56.491390 1588 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:13:56.492627 1588 hybrid_clock.cc:648] HybridClock initialized: now 1786126436492590 us; error 58 us; skew 500 ppm
I20260807 18:13:56.492961 1588 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:13:56 dist-test-slave-gkw7 krb5kdc[1446](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126436, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:13:56.496723 1588 init.cc:377] Logged in from keytab as kudu/127.18.154.1@KRBTEST.COM (short username kudu)
I20260807 18:13:56.498299 1588 webserver.cc:460] Webserver started at http://127.18.154.1:37631/ using document root <none> and password file <none>
I20260807 18:13:56.498975 1588 fs_manager.cc:359] Metadata directory not provided
I20260807 18:13:56.499074 1588 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:13:56.499351 1588 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:13:56.502502 1588 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126328808105-19048-0/minicluster-data/ts-0/data/instance:
uuid: "17f3613e4baa429a812f996d5d8219e7"
format_stamp: "Formatted at 2026-08-07 18:13:56 on dist-test-slave-gkw7"
server_key: "08f3f412f1dc49b1fe640da36a0ebcaf"
server_key_iv: "be4f30c685e31452323040d1d6692352"
server_key_version: "encryptionkey@0"
I20260807 18:13:56.503257 1588 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126328808105-19048-0/minicluster-data/ts-0/wal/instance:
uuid: "17f3613e4baa429a812f996d5d8219e7"
format_stamp: "Formatted at 2026-08-07 18:13:56 on dist-test-slave-gkw7"
server_key: "08f3f412f1dc49b1fe640da36a0ebcaf"
server_key_iv: "be4f30c685e31452323040d1d6692352"
server_key_version: "encryptionkey@0"
I20260807 18:13:56.508749 1588 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260807 18:13:56.512542 1614 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:13:56.514061 1588 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:13:56.514214 1588 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126328808105-19048-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126328808105-19048-0/minicluster-data/ts-0/wal
uuid: "17f3613e4baa429a812f996d5d8219e7"
format_stamp: "Formatted at 2026-08-07 18:13:56 on dist-test-slave-gkw7"
server_key: "08f3f412f1dc49b1fe640da36a0ebcaf"
server_key_iv: "be4f30c685e31452323040d1d6692352"
server_key_version: "encryptionkey@0"
I20260807 18:13:56.514379 1588 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126328808105-19048-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126328808105-19048-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126328808105-19048-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:13:56.538239 1588 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:13:56.542353 1588 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:13:56.542604 1588 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:13:56.543555 1588 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:13:56.544862 1588 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:13:56.544942 1588 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:13:56.545025 1588 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:13:56.545069 1588 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:13:56.579571 1588 rpc_server.cc:307] RPC server started. Bound to: 127.18.154.1:46729
I20260807 18:13:56.579622 1766 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.154.1:46729 every 8 connection(s)
I20260807 18:13:56.580974 1588 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126328808105-19048-0/minicluster-data/ts-0/data/info.pb
I20260807 18:13:56.589279 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 1588
I20260807 18:13:56.589557 19048 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126328808105-19048-0/minicluster-data/ts-0/wal/instance
I20260807 18:13:56.590390 19048 external_mini_cluster.cc:1428] Setting key 22d9de38dbf6639bd44e278940249685
Aug 07 18:13:56 dist-test-slave-gkw7 krb5kdc[1446](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126436, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.1@KRBTEST.COM for kudu/127.18.154.62@KRBTEST.COM
I20260807 18:13:56.602021 1770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:56.583734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.1:45057 (local address 127.18.154.62:37603)
0807 18:13:56.584199 (+ 465us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:56.584210 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:56.585191 (+ 981us) server_negotiation.cc:407] Connection header received
0807 18:13:56.587140 (+ 1949us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:56.587146 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:56.587250 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:56.587375 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:56.588918 (+ 1543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:56.589847 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:56.591625 (+ 1778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:56.592047 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:56.592189 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:56.595997 (+ 3808us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:13:56.596002 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:13:56.596005 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:13:56.596036 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:13:56.598789 (+ 2753us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:56.599553 (+ 764us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:56.599560 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:56.599563 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:56.599691 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:56.600148 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:56.600154 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:56.600157 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:56.600445 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:13:56.600606 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0807 18:13:56.601352 (+ 746us) server_negotiation.cc:299] Negotiation successful
0807 18:13:56.601617 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":348,"thread_start_us":223,"threads_started":1}
I20260807 18:13:56.603015 1771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:56.584289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:37603 (local address 127.18.154.1:45057)
0807 18:13:56.585006 (+ 717us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:56.585044 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0807 18:13:56.586859 (+ 1815us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:13:56.587615 (+ 756us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:13:56.587622 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:13:56.588133 (+ 511us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:13:56.588733 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:56.588758 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:56.590016 (+ 1258us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:56.590020 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:56.591467 (+ 1447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:56.591480 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:56.592281 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:56.592291 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:56.592553 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:56.593282 (+ 729us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:13:56.593304 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:13:56.595769 (+ 2465us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:13:56.598945 (+ 3176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:13:56.598955 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:13:56.598973 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:13:56.599426 (+ 453us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:13:56.599816 (+ 390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:13:56.599821 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:13:56.599825 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:13:56.600030 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:13:56.600578 (+ 548us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:13:56.600588 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:13:56.600950 (+ 362us) client_negotiation.cc:769] Sending connection context
0807 18:13:56.601249 (+ 299us) client_negotiation.cc:240] Negotiation successful
0807 18:13:56.601873 (+ 624us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":475,"thread_start_us":265,"threads_started":1}
I20260807 18:13:56.604748 1768 heartbeater.cc:347] Connected to a master server at 127.18.154.62:37603
I20260807 18:13:56.605098 1768 heartbeater.cc:464] Registering TS with master...
I20260807 18:13:56.606055 1768 heartbeater.cc:511] Master 127.18.154.62:37603 requested a full tablet report, sending...
I20260807 18:13:56.608610 1508 ts_manager.cc:194] Registered new tserver with Master: 17f3613e4baa429a812f996d5d8219e7 (127.18.154.1:46729)
I20260807 18:13:56.610911 1508 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.154.1@KRBTEST.COM'} at 127.18.154.1:45057
WARNING: no policy specified for kudu/127.18.154.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.154.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.154.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.2.keytab.
Entry for principal kudu/127.18.154.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.2.keytab.
I20260807 18:13:56.657434 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126328808105-19048-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126328808105-19048-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126328808105-19048-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126328808105-19048-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.154.2:0
--local_ip_for_outbound_sockets=127.18.154.2
--webserver_interface=127.18.154.2
--webserver_port=0
--tserver_master_addrs=127.18.154.62:37603
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.2.keytab
--principal=kudu/127.18.154.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.154.20:34025
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126328808105-19048-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126328808105-19048-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126328808105-19048-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:13:57.017680 1774 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:13:57.017973 1774 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:13:57.018078 1774 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:13:57.025365 1774 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:13:57.025477 1774 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:13:57.025664 1774 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.154.2
I20260807 18:13:57.035461 1774 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.154.20:34025
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126328808105-19048-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126328808105-19048-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.2.keytab
--principal=kudu/127.18.154.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.154.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126328808105-19048-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.154.2
--webserver_port=0
--tserver_master_addrs=127.18.154.62:37603
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.154.2
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126328808105-19048-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:13:57.036754 1774 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:13:57.038179 1774 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:13:57.052147 1786 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:13:57.052204 1781 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:13:57.052208 1783 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:13:57.055009 1774 server_base.cc:1034] running on GCE node
I20260807 18:13:57.055707 1774 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:13:57.056783 1774 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:13:57.057993 1774 hybrid_clock.cc:648] HybridClock initialized: now 1786126437057945 us; error 36 us; skew 500 ppm
I20260807 18:13:57.058367 1774 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:13:57 dist-test-slave-gkw7 krb5kdc[1446](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126437, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:13:57.061676 1774 init.cc:377] Logged in from keytab as kudu/127.18.154.2@KRBTEST.COM (short username kudu)
I20260807 18:13:57.063254 1774 webserver.cc:460] Webserver started at http://127.18.154.2:46415/ using document root <none> and password file <none>
I20260807 18:13:57.064023 1774 fs_manager.cc:359] Metadata directory not provided
I20260807 18:13:57.064193 1774 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:13:57.064517 1774 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:13:57.067698 1774 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126328808105-19048-0/minicluster-data/ts-1/data/instance:
uuid: "fed13caab5144cb69467a39c4c11037c"
format_stamp: "Formatted at 2026-08-07 18:13:57 on dist-test-slave-gkw7"
server_key: "e019497cc1cc64a1f9315ac20ea05b55"
server_key_iv: "f1025e6fd1bfb3fd0e81cfdf28251358"
server_key_version: "encryptionkey@0"
I20260807 18:13:57.068508 1774 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126328808105-19048-0/minicluster-data/ts-1/wal/instance:
uuid: "fed13caab5144cb69467a39c4c11037c"
format_stamp: "Formatted at 2026-08-07 18:13:57 on dist-test-slave-gkw7"
server_key: "e019497cc1cc64a1f9315ac20ea05b55"
server_key_iv: "f1025e6fd1bfb3fd0e81cfdf28251358"
server_key_version: "encryptionkey@0"
I20260807 18:13:57.074160 1774 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260807 18:13:57.077962 1795 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:13:57.079397 1774 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 18:13:57.079600 1774 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126328808105-19048-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126328808105-19048-0/minicluster-data/ts-1/wal
uuid: "fed13caab5144cb69467a39c4c11037c"
format_stamp: "Formatted at 2026-08-07 18:13:57 on dist-test-slave-gkw7"
server_key: "e019497cc1cc64a1f9315ac20ea05b55"
server_key_iv: "f1025e6fd1bfb3fd0e81cfdf28251358"
server_key_version: "encryptionkey@0"
I20260807 18:13:57.079821 1774 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126328808105-19048-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126328808105-19048-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126328808105-19048-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:13:57.105749 1774 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:13:57.110248 1774 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:13:57.110580 1774 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:13:57.111601 1774 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:13:57.113188 1774 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:13:57.113297 1774 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:13:57.113453 1774 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:13:57.113528 1774 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:13:57.149071 1774 rpc_server.cc:307] RPC server started. Bound to: 127.18.154.2:34637
I20260807 18:13:57.149286 1956 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.154.2:34637 every 8 connection(s)
I20260807 18:13:57.150555 1774 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126328808105-19048-0/minicluster-data/ts-1/data/info.pb
I20260807 18:13:57.157968 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 1774
I20260807 18:13:57.158171 19048 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126328808105-19048-0/minicluster-data/ts-1/wal/instance
I20260807 18:13:57.158612 19048 external_mini_cluster.cc:1428] Setting key ca336356ebe64e8bd31b70e8248a717f
Aug 07 18:13:57 dist-test-slave-gkw7 krb5kdc[1446](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126437, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.2@KRBTEST.COM for kudu/127.18.154.62@KRBTEST.COM
I20260807 18:13:57.174332 1962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:57.153230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.2:37399 (local address 127.18.154.62:37603)
0807 18:13:57.153739 (+ 509us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:57.153744 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:57.154743 (+ 999us) server_negotiation.cc:407] Connection header received
0807 18:13:57.156641 (+ 1898us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:57.156647 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:57.156747 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:57.156869 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:57.158629 (+ 1760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:57.159481 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:57.161719 (+ 2238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:57.162548 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:57.162700 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:57.166477 (+ 3777us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:13:57.166486 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:13:57.166493 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:13:57.166554 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:13:57.170432 (+ 3878us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:57.171363 (+ 931us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:57.171371 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:57.171377 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:57.171560 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:57.172076 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:57.172084 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:57.172089 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:57.172533 (+ 444us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:13:57.172694 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0807 18:13:57.173609 (+ 915us) server_negotiation.cc:299] Negotiation successful
0807 18:13:57.173869 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":389,"thread_start_us":228,"threads_started":1}
I20260807 18:13:57.175571 1964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:57.153829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:37603 (local address 127.18.154.2:37399)
0807 18:13:57.154567 (+ 738us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:57.154614 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 18:13:57.156357 (+ 1743us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:13:57.157125 (+ 768us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:13:57.157133 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:13:57.157750 (+ 617us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:13:57.158455 (+ 705us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:57.158474 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:57.159781 (+ 1307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:57.159787 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:57.161507 (+ 1720us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:57.161520 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:57.162777 (+ 1257us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:57.162782 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:57.162990 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:57.163731 (+ 741us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:13:57.163753 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:13:57.166199 (+ 2446us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:13:57.170611 (+ 4412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:13:57.170652 (+ 41us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:13:57.170667 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:13:57.171213 (+ 546us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:13:57.171715 (+ 502us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:13:57.171719 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:13:57.171722 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:13:57.171940 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:13:57.172712 (+ 772us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:13:57.172719 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:13:57.173123 (+ 404us) client_negotiation.cc:769] Sending connection context
0807 18:13:57.173459 (+ 336us) client_negotiation.cc:240] Negotiation successful
0807 18:13:57.174163 (+ 704us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":504,"thread_start_us":293,"threads_started":1}
I20260807 18:13:57.177449 1959 heartbeater.cc:347] Connected to a master server at 127.18.154.62:37603
I20260807 18:13:57.177870 1959 heartbeater.cc:464] Registering TS with master...
I20260807 18:13:57.178926 1959 heartbeater.cc:511] Master 127.18.154.62:37603 requested a full tablet report, sending...
I20260807 18:13:57.180980 1508 ts_manager.cc:194] Registered new tserver with Master: fed13caab5144cb69467a39c4c11037c (127.18.154.2:34637)
I20260807 18:13:57.182404 1508 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.154.2@KRBTEST.COM'} at 127.18.154.2:37399
WARNING: no policy specified for kudu/127.18.154.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.154.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.154.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.3.keytab.
Entry for principal kudu/127.18.154.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.3.keytab.
I20260807 18:13:57.229125 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126328808105-19048-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126328808105-19048-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126328808105-19048-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126328808105-19048-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.154.3:0
--local_ip_for_outbound_sockets=127.18.154.3
--webserver_interface=127.18.154.3
--webserver_port=0
--tserver_master_addrs=127.18.154.62:37603
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.3.keytab
--principal=kudu/127.18.154.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.154.20:34025
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126328808105-19048-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126328808105-19048-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126328808105-19048-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:13:57.579955 1967 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:13:57.580256 1967 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:13:57.580404 1967 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:13:57.588718 1967 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:13:57.588816 1967 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:13:57.588927 1967 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.154.3
I20260807 18:13:57.598969 1967 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.154.20:34025
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126328808105-19048-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126328808105-19048-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.3.keytab
--principal=kudu/127.18.154.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.154.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126328808105-19048-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.154.3
--webserver_port=0
--tserver_master_addrs=127.18.154.62:37603
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.154.3
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126328808105-19048-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:13:57.600224 1967 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:13:57.601471 1967 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 18:13:57.615002 1768 heartbeater.cc:503] Master 127.18.154.62:37603 was elected leader, sending a full tablet report...
W20260807 18:13:57.615245 1975 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:13:57.615252 1976 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:13:57.615765 1979 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:13:57.616648 1967 server_base.cc:1034] running on GCE node
I20260807 18:13:57.617468 1967 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:13:57.618602 1967 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:13:57.619809 1967 hybrid_clock.cc:648] HybridClock initialized: now 1786126437619755 us; error 41 us; skew 500 ppm
I20260807 18:13:57.620144 1967 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:13:57 dist-test-slave-gkw7 krb5kdc[1446](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126437, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:13:57.623389 1967 init.cc:377] Logged in from keytab as kudu/127.18.154.3@KRBTEST.COM (short username kudu)
I20260807 18:13:57.624958 1967 webserver.cc:460] Webserver started at http://127.18.154.3:40315/ using document root <none> and password file <none>
I20260807 18:13:57.625797 1967 fs_manager.cc:359] Metadata directory not provided
I20260807 18:13:57.625954 1967 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:13:57.626286 1967 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:13:57.629956 1967 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126328808105-19048-0/minicluster-data/ts-2/data/instance:
uuid: "84917a8576f442039a9416004e1fcad7"
format_stamp: "Formatted at 2026-08-07 18:13:57 on dist-test-slave-gkw7"
server_key: "434ee6e6bc6ba0de935db437664cd7ce"
server_key_iv: "47b91836e7ace052ff8a06c43655c896"
server_key_version: "encryptionkey@0"
I20260807 18:13:57.630856 1967 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126328808105-19048-0/minicluster-data/ts-2/wal/instance:
uuid: "84917a8576f442039a9416004e1fcad7"
format_stamp: "Formatted at 2026-08-07 18:13:57 on dist-test-slave-gkw7"
server_key: "434ee6e6bc6ba0de935db437664cd7ce"
server_key_iv: "47b91836e7ace052ff8a06c43655c896"
server_key_version: "encryptionkey@0"
I20260807 18:13:57.636879 1967 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260807 18:13:57.640794 1992 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:13:57.642257 1967 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260807 18:13:57.642444 1967 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126328808105-19048-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126328808105-19048-0/minicluster-data/ts-2/wal
uuid: "84917a8576f442039a9416004e1fcad7"
format_stamp: "Formatted at 2026-08-07 18:13:57 on dist-test-slave-gkw7"
server_key: "434ee6e6bc6ba0de935db437664cd7ce"
server_key_iv: "47b91836e7ace052ff8a06c43655c896"
server_key_version: "encryptionkey@0"
I20260807 18:13:57.642663 1967 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126328808105-19048-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126328808105-19048-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126328808105-19048-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:13:57.662278 1967 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:13:57.666646 1967 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:13:57.666965 1967 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:13:57.667999 1967 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:13:57.669492 1967 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:13:57.669648 1967 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260807 18:13:57.669809 1967 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:13:57.669883 1967 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:13:57.705721 1967 rpc_server.cc:307] RPC server started. Bound to: 127.18.154.3:41403
I20260807 18:13:57.705741 2161 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.154.3:41403 every 8 connection(s)
I20260807 18:13:57.707087 1967 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126328808105-19048-0/minicluster-data/ts-2/data/info.pb
I20260807 18:13:57.709262 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 1967
I20260807 18:13:57.709446 19048 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126328808105-19048-0/minicluster-data/ts-2/wal/instance
I20260807 18:13:57.710085 19048 external_mini_cluster.cc:1428] Setting key 6964cccc96418af4b9779e1d4c66fde4
Aug 07 18:13:57 dist-test-slave-gkw7 krb5kdc[1446](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126437, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.3@KRBTEST.COM for kudu/127.18.154.62@KRBTEST.COM
I20260807 18:13:57.728338 2164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:57.709867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.3:45709 (local address 127.18.154.62:37603)
0807 18:13:57.710365 (+ 498us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:57.710373 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:57.711549 (+ 1176us) server_negotiation.cc:407] Connection header received
0807 18:13:57.713676 (+ 2127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:57.713681 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:57.713750 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:57.713843 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:57.715400 (+ 1557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:57.716209 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:57.717863 (+ 1654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:57.718269 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:57.718383 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:57.721970 (+ 3587us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:13:57.721975 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:13:57.721978 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:13:57.722021 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:13:57.724939 (+ 2918us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:57.725827 (+ 888us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:57.725835 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:57.725839 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:57.726000 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:57.726494 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:57.726499 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:57.726503 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:57.726802 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:13:57.726979 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0807 18:13:57.727723 (+ 744us) server_negotiation.cc:299] Negotiation successful
0807 18:13:57.727951 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":362,"thread_start_us":241,"threads_started":1}
I20260807 18:13:57.729367 2167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:57.710481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:37603 (local address 127.18.154.3:45709)
0807 18:13:57.711373 (+ 892us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:57.711413 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 18:13:57.713369 (+ 1956us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:13:57.714110 (+ 741us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:13:57.714118 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:13:57.714642 (+ 524us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:13:57.715241 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:57.715259 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:57.716405 (+ 1146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:57.716409 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:57.717735 (+ 1326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:57.717747 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:57.718519 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:57.718525 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:57.718738 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:57.719429 (+ 691us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:13:57.719457 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:13:57.721747 (+ 2290us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:13:57.725082 (+ 3335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:13:57.725090 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:13:57.725104 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:13:57.725696 (+ 592us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:13:57.726156 (+ 460us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:13:57.726166 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:13:57.726170 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:13:57.726367 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:13:57.726940 (+ 573us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:13:57.726950 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:13:57.727318 (+ 368us) client_negotiation.cc:769] Sending connection context
0807 18:13:57.727611 (+ 293us) client_negotiation.cc:240] Negotiation successful
0807 18:13:57.728168 (+ 557us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":601,"thread_start_us":360,"threads_started":1}
I20260807 18:13:57.730785 2162 heartbeater.cc:347] Connected to a master server at 127.18.154.62:37603
I20260807 18:13:57.731096 2162 heartbeater.cc:464] Registering TS with master...
I20260807 18:13:57.731832 2162 heartbeater.cc:511] Master 127.18.154.62:37603 requested a full tablet report, sending...
I20260807 18:13:57.733219 1508 ts_manager.cc:194] Registered new tserver with Master: 84917a8576f442039a9416004e1fcad7 (127.18.154.3:41403)
I20260807 18:13:57.734056 1508 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.154.3@KRBTEST.COM'} at 127.18.154.3:45709
I20260807 18:13:57.738315 19048 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 18:13:57.756012 2164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:57.743036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50194 (local address 127.18.154.62:37603)
0807 18:13:57.743260 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:57.743268 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:57.743657 (+ 389us) server_negotiation.cc:407] Connection header received
0807 18:13:57.743867 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:57.743871 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:57.743932 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:57.744033 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:57.745596 (+ 1563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:57.746364 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:57.747675 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:57.748119 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:57.748259 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:57.750659 (+ 2400us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:13:57.750664 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:13:57.750668 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:13:57.750699 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:13:57.753260 (+ 2561us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:57.753975 (+ 715us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:57.753982 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:57.753985 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:57.754117 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:57.754502 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:57.754508 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:57.754511 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:57.754851 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:13:57.755117 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0807 18:13:57.755456 (+ 339us) server_negotiation.cc:299] Negotiation successful
0807 18:13:57.755679 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 18:13:57.760335 1508 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50194:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 18:13:57.767253 1508 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:13:57.793069 2180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:57.779518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40454 (local address 127.18.154.2:34637)
0807 18:13:57.780043 (+ 525us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:57.780052 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:57.780890 (+ 838us) server_negotiation.cc:407] Connection header received
0807 18:13:57.781377 (+ 487us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:57.781390 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:57.781750 (+ 360us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:57.781968 (+ 218us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:13:57.783182 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:57.785247 (+ 2065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:57.788948 (+ 3701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:57.790982 (+ 2034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:57.791128 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:57.791398 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0807 18:13:57.792291 (+ 893us) server_negotiation.cc:299] Negotiation successful
0807 18:13:57.792548 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":358,"thread_start_us":415,"threads_started":1}
I20260807 18:13:57.794251 2179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:57.779279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.2:34637 (local address 127.0.0.1:40454)
0807 18:13:57.780730 (+ 1451us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:57.780771 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 18:13:57.781084 (+ 313us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:13:57.782348 (+ 1264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:13:57.782359 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:13:57.782400 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:13:57.782924 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:57.782940 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:57.785481 (+ 2541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:57.785489 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:57.788696 (+ 3207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:57.788727 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:57.791641 (+ 2914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:57.791651 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:57.791961 (+ 310us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:57.792032 (+ 71us) client_negotiation.cc:769] Sending connection context
0807 18:13:57.793425 (+ 1393us) client_negotiation.cc:240] Negotiation successful
0807 18:13:57.793698 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1196,"thread_start_us":1035,"threads_started":1}
I20260807 18:13:57.800007 1864 tablet_service.cc:1467] Processing CreateTablet for tablet a271f83fa2a44020b676aa92f5c2e907 (DEFAULT_TABLE table=test-table [id=4041e929e3064654af3f8f0a47e255f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:13:57.801466 1864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a271f83fa2a44020b676aa92f5c2e907. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:13:57.804529 2179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:57.781265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.3:41403 (local address 127.0.0.1:56442)
0807 18:13:57.794641 (+ 13376us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:57.794665 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 18:13:57.794804 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:13:57.796428 (+ 1624us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:13:57.796433 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:13:57.797277 (+ 844us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:13:57.797916 (+ 639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:57.797937 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:57.799574 (+ 1637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:57.799580 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:57.801737 (+ 2157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:57.801752 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:57.803600 (+ 1848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:57.803606 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:57.803819 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:57.803854 (+ 35us) client_negotiation.cc:769] Sending connection context
0807 18:13:57.803952 (+ 98us) client_negotiation.cc:240] Negotiation successful
0807 18:13:57.804114 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13244,"mutex_wait_us":56,"thread_start_us":13970,"threads_started":1}
I20260807 18:13:57.804608 2184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:57.782123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56442 (local address 127.18.154.3:41403)
0807 18:13:57.795932 (+ 13809us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:57.795938 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:57.795967 (+ 29us) server_negotiation.cc:407] Connection header received
0807 18:13:57.796043 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:57.796048 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:57.796241 (+ 193us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:57.796442 (+ 201us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:13:57.798135 (+ 1693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:57.799424 (+ 1289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:57.801951 (+ 2527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:57.803401 (+ 1450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:57.803570 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:57.803858 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0807 18:13:57.804043 (+ 185us) server_negotiation.cc:299] Negotiation successful
0807 18:13:57.804251 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13572,"spinlock_wait_cycles":24448,"thread_start_us":13532,"threads_started":1}
I20260807 18:13:57.805218 2181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:57.779958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.1:46729 (local address 127.0.0.1:42330)
0807 18:13:57.794447 (+ 14489us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:57.794474 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0807 18:13:57.794616 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:13:57.796880 (+ 2264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:13:57.796885 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:13:57.796909 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:13:57.797832 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:57.797847 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:57.799794 (+ 1947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:57.799801 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:57.802312 (+ 2511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:57.802328 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:57.804337 (+ 2009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:57.804346 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:57.804552 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:57.804593 (+ 41us) client_negotiation.cc:769] Sending connection context
0807 18:13:57.804669 (+ 76us) client_negotiation.cc:240] Negotiation successful
0807 18:13:57.804802 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14320,"mutex_wait_us":262,"thread_start_us":14747,"threads_started":1}
I20260807 18:13:57.806408 2182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:57.780648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42330 (local address 127.18.154.1:46729)
0807 18:13:57.796234 (+ 15586us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:57.796243 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:57.796271 (+ 28us) server_negotiation.cc:407] Connection header received
0807 18:13:57.796369 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:57.796377 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:57.796667 (+ 290us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:57.796848 (+ 181us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:13:57.798081 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:57.799648 (+ 1567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:57.802509 (+ 2861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:57.804077 (+ 1568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:57.805574 (+ 1497us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:57.805818 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0807 18:13:57.805904 (+ 86us) server_negotiation.cc:299] Negotiation successful
0807 18:13:57.806079 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15397,"thread_start_us":16464,"threads_started":1}
I20260807 18:13:57.809285 2061 tablet_service.cc:1467] Processing CreateTablet for tablet a271f83fa2a44020b676aa92f5c2e907 (DEFAULT_TABLE table=test-table [id=4041e929e3064654af3f8f0a47e255f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:13:57.809775 1678 tablet_service.cc:1467] Processing CreateTablet for tablet a271f83fa2a44020b676aa92f5c2e907 (DEFAULT_TABLE table=test-table [id=4041e929e3064654af3f8f0a47e255f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:13:57.810861 2061 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a271f83fa2a44020b676aa92f5c2e907. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:13:57.811199 1678 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a271f83fa2a44020b676aa92f5c2e907. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:13:57.818025 2185 tablet_bootstrap.cc:492] T a271f83fa2a44020b676aa92f5c2e907 P fed13caab5144cb69467a39c4c11037c: Bootstrap starting.
I20260807 18:13:57.821885 2185 tablet_bootstrap.cc:654] T a271f83fa2a44020b676aa92f5c2e907 P fed13caab5144cb69467a39c4c11037c: Neither blocks nor log segments found. Creating new log.
I20260807 18:13:57.823289 2185 log.cc:824] T a271f83fa2a44020b676aa92f5c2e907 P fed13caab5144cb69467a39c4c11037c: Log is configured to *not* fsync() on all Append() calls
I20260807 18:13:57.823925 2186 tablet_bootstrap.cc:492] T a271f83fa2a44020b676aa92f5c2e907 P 84917a8576f442039a9416004e1fcad7: Bootstrap starting.
I20260807 18:13:57.827134 2186 tablet_bootstrap.cc:654] T a271f83fa2a44020b676aa92f5c2e907 P 84917a8576f442039a9416004e1fcad7: Neither blocks nor log segments found. Creating new log.
I20260807 18:13:57.828011 2187 tablet_bootstrap.cc:492] T a271f83fa2a44020b676aa92f5c2e907 P 17f3613e4baa429a812f996d5d8219e7: Bootstrap starting.
I20260807 18:13:57.829183 2186 log.cc:824] T a271f83fa2a44020b676aa92f5c2e907 P 84917a8576f442039a9416004e1fcad7: Log is configured to *not* fsync() on all Append() calls
I20260807 18:13:57.831938 2187 tablet_bootstrap.cc:654] T a271f83fa2a44020b676aa92f5c2e907 P 17f3613e4baa429a812f996d5d8219e7: Neither blocks nor log segments found. Creating new log.
I20260807 18:13:57.832427 2186 tablet_bootstrap.cc:492] T a271f83fa2a44020b676aa92f5c2e907 P 84917a8576f442039a9416004e1fcad7: No bootstrap required, opened a new log
I20260807 18:13:57.832743 2185 tablet_bootstrap.cc:492] T a271f83fa2a44020b676aa92f5c2e907 P fed13caab5144cb69467a39c4c11037c: No bootstrap required, opened a new log
I20260807 18:13:57.832767 2186 ts_tablet_manager.cc:1397] T a271f83fa2a44020b676aa92f5c2e907 P 84917a8576f442039a9416004e1fcad7: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.003s
I20260807 18:13:57.833051 2185 ts_tablet_manager.cc:1397] T a271f83fa2a44020b676aa92f5c2e907 P fed13caab5144cb69467a39c4c11037c: Time spent bootstrapping tablet: real 0.015s user 0.006s sys 0.007s
I20260807 18:13:57.833385 2187 log.cc:824] T a271f83fa2a44020b676aa92f5c2e907 P 17f3613e4baa429a812f996d5d8219e7: Log is configured to *not* fsync() on all Append() calls
I20260807 18:13:57.837041 2187 tablet_bootstrap.cc:492] T a271f83fa2a44020b676aa92f5c2e907 P 17f3613e4baa429a812f996d5d8219e7: No bootstrap required, opened a new log
I20260807 18:13:57.837386 2187 ts_tablet_manager.cc:1397] T a271f83fa2a44020b676aa92f5c2e907 P 17f3613e4baa429a812f996d5d8219e7: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.003s
I20260807 18:13:57.841485 2186 raft_consensus.cc:348] T a271f83fa2a44020b676aa92f5c2e907 P 84917a8576f442039a9416004e1fcad7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fed13caab5144cb69467a39c4c11037c" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 34637 } } peers { permanent_uuid: "84917a8576f442039a9416004e1fcad7" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 41403 } } peers { permanent_uuid: "17f3613e4baa429a812f996d5d8219e7" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 46729 } }
I20260807 18:13:57.841998 2186 raft_consensus.cc:374] T a271f83fa2a44020b676aa92f5c2e907 P 84917a8576f442039a9416004e1fcad7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:13:57.842168 2186 raft_consensus.cc:729] T a271f83fa2a44020b676aa92f5c2e907 P 84917a8576f442039a9416004e1fcad7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84917a8576f442039a9416004e1fcad7, State: Initialized, Role: FOLLOWER
I20260807 18:13:57.842871 2186 consensus_queue.cc:260] T a271f83fa2a44020b676aa92f5c2e907 P 84917a8576f442039a9416004e1fcad7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fed13caab5144cb69467a39c4c11037c" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 34637 } } peers { permanent_uuid: "84917a8576f442039a9416004e1fcad7" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 41403 } } peers { permanent_uuid: "17f3613e4baa429a812f996d5d8219e7" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 46729 } }
I20260807 18:13:57.844179 2185 raft_consensus.cc:348] T a271f83fa2a44020b676aa92f5c2e907 P fed13caab5144cb69467a39c4c11037c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fed13caab5144cb69467a39c4c11037c" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 34637 } } peers { permanent_uuid: "84917a8576f442039a9416004e1fcad7" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 41403 } } peers { permanent_uuid: "17f3613e4baa429a812f996d5d8219e7" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 46729 } }
I20260807 18:13:57.844748 2185 raft_consensus.cc:374] T a271f83fa2a44020b676aa92f5c2e907 P fed13caab5144cb69467a39c4c11037c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:13:57.844897 2185 raft_consensus.cc:729] T a271f83fa2a44020b676aa92f5c2e907 P fed13caab5144cb69467a39c4c11037c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fed13caab5144cb69467a39c4c11037c, State: Initialized, Role: FOLLOWER
I20260807 18:13:57.845791 2185 consensus_queue.cc:260] T a271f83fa2a44020b676aa92f5c2e907 P fed13caab5144cb69467a39c4c11037c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fed13caab5144cb69467a39c4c11037c" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 34637 } } peers { permanent_uuid: "84917a8576f442039a9416004e1fcad7" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 41403 } } peers { permanent_uuid: "17f3613e4baa429a812f996d5d8219e7" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 46729 } }
I20260807 18:13:57.846179 2162 heartbeater.cc:503] Master 127.18.154.62:37603 was elected leader, sending a full tablet report...
I20260807 18:13:57.846983 2186 ts_tablet_manager.cc:1428] T a271f83fa2a44020b676aa92f5c2e907 P 84917a8576f442039a9416004e1fcad7: Time spent starting tablet: real 0.014s user 0.007s sys 0.007s
I20260807 18:13:57.849773 2187 raft_consensus.cc:348] T a271f83fa2a44020b676aa92f5c2e907 P 17f3613e4baa429a812f996d5d8219e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fed13caab5144cb69467a39c4c11037c" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 34637 } } peers { permanent_uuid: "84917a8576f442039a9416004e1fcad7" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 41403 } } peers { permanent_uuid: "17f3613e4baa429a812f996d5d8219e7" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 46729 } }
I20260807 18:13:57.850351 2187 raft_consensus.cc:374] T a271f83fa2a44020b676aa92f5c2e907 P 17f3613e4baa429a812f996d5d8219e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:13:57.850402 1959 heartbeater.cc:503] Master 127.18.154.62:37603 was elected leader, sending a full tablet report...
I20260807 18:13:57.850517 2187 raft_consensus.cc:729] T a271f83fa2a44020b676aa92f5c2e907 P 17f3613e4baa429a812f996d5d8219e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17f3613e4baa429a812f996d5d8219e7, State: Initialized, Role: FOLLOWER
I20260807 18:13:57.851382 2185 ts_tablet_manager.cc:1428] T a271f83fa2a44020b676aa92f5c2e907 P fed13caab5144cb69467a39c4c11037c: Time spent starting tablet: real 0.018s user 0.017s sys 0.000s
I20260807 18:13:57.851217 2187 consensus_queue.cc:260] T a271f83fa2a44020b676aa92f5c2e907 P 17f3613e4baa429a812f996d5d8219e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fed13caab5144cb69467a39c4c11037c" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 34637 } } peers { permanent_uuid: "84917a8576f442039a9416004e1fcad7" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 41403 } } peers { permanent_uuid: "17f3613e4baa429a812f996d5d8219e7" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 46729 } }
I20260807 18:13:57.853137 2187 ts_tablet_manager.cc:1428] T a271f83fa2a44020b676aa92f5c2e907 P 17f3613e4baa429a812f996d5d8219e7: Time spent starting tablet: real 0.016s user 0.008s sys 0.008s
W20260807 18:13:57.902689 1960 tablet.cc:2369] T a271f83fa2a44020b676aa92f5c2e907 P fed13caab5144cb69467a39c4c11037c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 18:13:57.958896 2163 tablet.cc:2369] T a271f83fa2a44020b676aa92f5c2e907 P 84917a8576f442039a9416004e1fcad7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:13:58.024151 2195 raft_consensus.cc:482] T a271f83fa2a44020b676aa92f5c2e907 P 17f3613e4baa429a812f996d5d8219e7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:13:58.024580 2195 raft_consensus.cc:504] T a271f83fa2a44020b676aa92f5c2e907 P 17f3613e4baa429a812f996d5d8219e7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fed13caab5144cb69467a39c4c11037c" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 34637 } } peers { permanent_uuid: "84917a8576f442039a9416004e1fcad7" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 41403 } } peers { permanent_uuid: "17f3613e4baa429a812f996d5d8219e7" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 46729 } }
I20260807 18:13:58.026600 2195 leader_election.cc:288] T a271f83fa2a44020b676aa92f5c2e907 P 17f3613e4baa429a812f996d5d8219e7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fed13caab5144cb69467a39c4c11037c (127.18.154.2:34637), 84917a8576f442039a9416004e1fcad7 (127.18.154.3:41403)
I20260807 18:13:58.034896 2184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:58.028039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.1:43545 (local address 127.18.154.3:41403)
0807 18:13:58.028251 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:58.028259 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:58.028503 (+ 244us) server_negotiation.cc:407] Connection header received
0807 18:13:58.028627 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:58.028635 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:58.028722 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:58.028829 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:13:58.029504 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:58.030436 (+ 932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:58.032485 (+ 2049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:58.033730 (+ 1245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:58.033868 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:58.033933 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0807 18:13:58.034471 (+ 538us) server_negotiation.cc:299] Negotiation successful
0807 18:13:58.034615 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260807 18:13:58.034894 2180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:58.027199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.1:53751 (local address 127.18.154.2:34637)
0807 18:13:58.027472 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:58.027478 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:58.027793 (+ 315us) server_negotiation.cc:407] Connection header received
0807 18:13:58.028111 (+ 318us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:58.028120 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:58.028214 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:58.028292 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:13:58.029249 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:58.030133 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:58.032358 (+ 2225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:58.033676 (+ 1318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:58.033851 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:58.033927 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0807 18:13:58.034471 (+ 544us) server_negotiation.cc:299] Negotiation successful
0807 18:13:58.034615 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260807 18:13:58.035110 2199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:58.026947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.2:34637 (local address 127.18.154.1:53751)
0807 18:13:58.027638 (+ 691us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:58.027656 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 18:13:58.027868 (+ 212us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:13:58.028539 (+ 671us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:13:58.028545 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:13:58.028581 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:13:58.029079 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:58.029092 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:58.030270 (+ 1178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:58.030276 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:58.032192 (+ 1916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:58.032204 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:58.033959 (+ 1755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:58.033967 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:58.034289 (+ 322us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:58.034313 (+ 24us) client_negotiation.cc:769] Sending connection context
0807 18:13:58.034415 (+ 102us) client_negotiation.cc:240] Negotiation successful
0807 18:13:58.034622 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":565,"mutex_wait_us":143,"thread_start_us":340,"threads_started":1}
I20260807 18:13:58.035133 2200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:58.027714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.3:41403 (local address 127.18.154.1:43545)
0807 18:13:58.028391 (+ 677us) negotiation.cc:107] Waiting for socket to connect
0807 18:13:58.028406 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 18:13:58.028515 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:13:58.028847 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:13:58.028850 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:13:58.028867 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:13:58.029322 (+ 455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:58.029335 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:58.030588 (+ 1253us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:58.030595 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:58.032259 (+ 1664us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:13:58.032274 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:13:58.033975 (+ 1701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:13:58.033980 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:13:58.034256 (+ 276us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:58.034313 (+ 57us) client_negotiation.cc:769] Sending connection context
0807 18:13:58.034415 (+ 102us) client_negotiation.cc:240] Negotiation successful
0807 18:13:58.034622 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":582,"mutex_wait_us":52,"thread_start_us":330,"threads_started":1}
I20260807 18:13:58.036264 2094 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a271f83fa2a44020b676aa92f5c2e907" candidate_uuid: "17f3613e4baa429a812f996d5d8219e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84917a8576f442039a9416004e1fcad7" is_pre_election: true
I20260807 18:13:58.036315 1891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a271f83fa2a44020b676aa92f5c2e907" candidate_uuid: "17f3613e4baa429a812f996d5d8219e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fed13caab5144cb69467a39c4c11037c" is_pre_election: true
I20260807 18:13:58.036808 2094 raft_consensus.cc:2454] T a271f83fa2a44020b676aa92f5c2e907 P 84917a8576f442039a9416004e1fcad7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17f3613e4baa429a812f996d5d8219e7 in term 0.
I20260807 18:13:58.036865 1891 raft_consensus.cc:2454] T a271f83fa2a44020b676aa92f5c2e907 P fed13caab5144cb69467a39c4c11037c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17f3613e4baa429a812f996d5d8219e7 in term 0.
I20260807 18:13:58.037495 1618 leader_election.cc:302] T a271f83fa2a44020b676aa92f5c2e907 P 17f3613e4baa429a812f996d5d8219e7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17f3613e4baa429a812f996d5d8219e7, 84917a8576f442039a9416004e1fcad7; no voters:
I20260807 18:13:58.038012 2195 raft_consensus.cc:2781] T a271f83fa2a44020b676aa92f5c2e907 P 17f3613e4baa429a812f996d5d8219e7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:13:58.038185 2195 raft_consensus.cc:482] T a271f83fa2a44020b676aa92f5c2e907 P 17f3613e4baa429a812f996d5d8219e7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:13:58.038329 2195 raft_consensus.cc:3037] T a271f83fa2a44020b676aa92f5c2e907 P 17f3613e4baa429a812f996d5d8219e7 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:13:58.040724 2195 raft_consensus.cc:504] T a271f83fa2a44020b676aa92f5c2e907 P 17f3613e4baa429a812f996d5d8219e7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fed13caab5144cb69467a39c4c11037c" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 34637 } } peers { permanent_uuid: "84917a8576f442039a9416004e1fcad7" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 41403 } } peers { permanent_uuid: "17f3613e4baa429a812f996d5d8219e7" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 46729 } }
I20260807 18:13:58.041447 2195 leader_election.cc:288] T a271f83fa2a44020b676aa92f5c2e907 P 17f3613e4baa429a812f996d5d8219e7 [CANDIDATE]: Term 1 election: Requested vote from peers fed13caab5144cb69467a39c4c11037c (127.18.154.2:34637), 84917a8576f442039a9416004e1fcad7 (127.18.154.3:41403)
I20260807 18:13:58.042021 2094 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a271f83fa2a44020b676aa92f5c2e907" candidate_uuid: "17f3613e4baa429a812f996d5d8219e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84917a8576f442039a9416004e1fcad7"
I20260807 18:13:58.042040 1891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a271f83fa2a44020b676aa92f5c2e907" candidate_uuid: "17f3613e4baa429a812f996d5d8219e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fed13caab5144cb69467a39c4c11037c"
I20260807 18:13:58.042238 2094 raft_consensus.cc:3037] T a271f83fa2a44020b676aa92f5c2e907 P 84917a8576f442039a9416004e1fcad7 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:13:58.042243 1891 raft_consensus.cc:3037] T a271f83fa2a44020b676aa92f5c2e907 P fed13caab5144cb69467a39c4c11037c [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:13:58.044461 2094 raft_consensus.cc:2454] T a271f83fa2a44020b676aa92f5c2e907 P 84917a8576f442039a9416004e1fcad7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17f3613e4baa429a812f996d5d8219e7 in term 1.
I20260807 18:13:58.044471 1891 raft_consensus.cc:2454] T a271f83fa2a44020b676aa92f5c2e907 P fed13caab5144cb69467a39c4c11037c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17f3613e4baa429a812f996d5d8219e7 in term 1.
I20260807 18:13:58.044890 1618 leader_election.cc:302] T a271f83fa2a44020b676aa92f5c2e907 P 17f3613e4baa429a812f996d5d8219e7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17f3613e4baa429a812f996d5d8219e7, 84917a8576f442039a9416004e1fcad7; no voters:
I20260807 18:13:58.045197 2195 raft_consensus.cc:2781] T a271f83fa2a44020b676aa92f5c2e907 P 17f3613e4baa429a812f996d5d8219e7 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:13:58.045740 2195 raft_consensus.cc:686] T a271f83fa2a44020b676aa92f5c2e907 P 17f3613e4baa429a812f996d5d8219e7 [term 1 LEADER]: Becoming Leader. State: Replica: 17f3613e4baa429a812f996d5d8219e7, State: Running, Role: LEADER
I20260807 18:13:58.046548 2195 consensus_queue.cc:237] T a271f83fa2a44020b676aa92f5c2e907 P 17f3613e4baa429a812f996d5d8219e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fed13caab5144cb69467a39c4c11037c" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 34637 } } peers { permanent_uuid: "84917a8576f442039a9416004e1fcad7" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 41403 } } peers { permanent_uuid: "17f3613e4baa429a812f996d5d8219e7" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 46729 } }
I20260807 18:13:58.051522 1507 catalog_manager.cc:5495] T a271f83fa2a44020b676aa92f5c2e907 P 17f3613e4baa429a812f996d5d8219e7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 17f3613e4baa429a812f996d5d8219e7 (127.18.154.1). New cstate: current_term: 1 leader_uuid: "17f3613e4baa429a812f996d5d8219e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fed13caab5144cb69467a39c4c11037c" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 34637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "84917a8576f442039a9416004e1fcad7" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 41403 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "17f3613e4baa429a812f996d5d8219e7" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 46729 } health_report { overall_health: HEALTHY } } }
W20260807 18:13:58.083645 1769 tablet.cc:2369] T a271f83fa2a44020b676aa92f5c2e907 P 17f3613e4baa429a812f996d5d8219e7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:13:58.103627 2182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:58.097357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42342 (local address 127.18.154.1:46729)
0807 18:13:58.097631 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:58.097640 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:58.097726 (+ 86us) server_negotiation.cc:407] Connection header received
0807 18:13:58.098015 (+ 289us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:58.098020 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:58.098120 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:58.098300 (+ 180us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:13:58.098726 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:58.099552 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:58.101076 (+ 1524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:58.101625 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:58.101770 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:58.102381 (+ 611us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:13:58.102960 (+ 579us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:13:58.103146 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0807 18:13:58.103217 (+ 71us) server_negotiation.cc:299] Negotiation successful
0807 18:13:58.103313 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260807 18:13:58.112363 2094 raft_consensus.cc:1261] T a271f83fa2a44020b676aa92f5c2e907 P 84917a8576f442039a9416004e1fcad7 [term 1 FOLLOWER]: Refusing update from remote peer 17f3613e4baa429a812f996d5d8219e7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:13:58.112409 1891 raft_consensus.cc:1261] T a271f83fa2a44020b676aa92f5c2e907 P fed13caab5144cb69467a39c4c11037c [term 1 FOLLOWER]: Refusing update from remote peer 17f3613e4baa429a812f996d5d8219e7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:13:58.113410 2195 consensus_queue.cc:1035] T a271f83fa2a44020b676aa92f5c2e907 P 17f3613e4baa429a812f996d5d8219e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fed13caab5144cb69467a39c4c11037c" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 34637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:13:58.113788 2202 consensus_queue.cc:1035] T a271f83fa2a44020b676aa92f5c2e907 P 17f3613e4baa429a812f996d5d8219e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84917a8576f442039a9416004e1fcad7" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 41403 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:13:58.125926 2210 mvcc.cc:205] Tried to move back new op lower bound from 7315973890497728512 to 7315973890242834432. Current Snapshot: MvccSnapshot[applied={T|T < 7315973890497728512}]
I20260807 18:13:58.131685 2209 mvcc.cc:205] Tried to move back new op lower bound from 7315973890497728512 to 7315973890242834432. Current Snapshot: MvccSnapshot[applied={T|T < 7315973890497728512}]
I20260807 18:13:58.145310 1507 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50194:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:13:58.145713 1507 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50194:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:13:58.150202 1507 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a5fd322e4dd64f8eb8309524ebd988c2: Sending DeleteTablet for 3 replicas of tablet a271f83fa2a44020b676aa92f5c2e907
I20260807 18:13:58.151530 2061 tablet_service.cc:1514] Processing DeleteTablet for tablet a271f83fa2a44020b676aa92f5c2e907 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:13:58 UTC) from {username='kudu', principal='kudu/127.18.154.62@KRBTEST.COM'} at 127.0.0.1:56442
I20260807 18:13:58.151539 1678 tablet_service.cc:1514] Processing DeleteTablet for tablet a271f83fa2a44020b676aa92f5c2e907 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:13:58 UTC) from {username='kudu', principal='kudu/127.18.154.62@KRBTEST.COM'} at 127.0.0.1:42330
I20260807 18:13:58.152601 1864 tablet_service.cc:1514] Processing DeleteTablet for tablet a271f83fa2a44020b676aa92f5c2e907 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:13:58 UTC) from {username='kudu', principal='kudu/127.18.154.62@KRBTEST.COM'} at 127.0.0.1:40454
I20260807 18:13:58.154367 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 1588
I20260807 18:13:58.171765 2222 tablet_replica.cc:330] T a271f83fa2a44020b676aa92f5c2e907 P 17f3613e4baa429a812f996d5d8219e7: stopping tablet replica
I20260807 18:13:58.172317 2222 raft_consensus.cc:2229] T a271f83fa2a44020b676aa92f5c2e907 P 17f3613e4baa429a812f996d5d8219e7 [term 1 LEADER]: Raft consensus shutting down.
I20260807 18:13:58.173092 2222 raft_consensus.cc:2258] T a271f83fa2a44020b676aa92f5c2e907 P 17f3613e4baa429a812f996d5d8219e7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:13:58.176695 2222 ts_tablet_manager.cc:1905] T a271f83fa2a44020b676aa92f5c2e907 P 17f3613e4baa429a812f996d5d8219e7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:13:58.184011 2221 tablet_replica.cc:330] T a271f83fa2a44020b676aa92f5c2e907 P 84917a8576f442039a9416004e1fcad7: stopping tablet replica
I20260807 18:13:58.185613 2221 raft_consensus.cc:2229] T a271f83fa2a44020b676aa92f5c2e907 P 84917a8576f442039a9416004e1fcad7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:13:58.188449 2222 ts_tablet_manager.cc:1918] T a271f83fa2a44020b676aa92f5c2e907 P 17f3613e4baa429a812f996d5d8219e7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:13:58.188724 2222 log.cc:1196] T a271f83fa2a44020b676aa92f5c2e907 P 17f3613e4baa429a812f996d5d8219e7: Deleting WAL directory at /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126328808105-19048-0/minicluster-data/ts-0/wal/wals/a271f83fa2a44020b676aa92f5c2e907
I20260807 18:13:58.189393 2222 ts_tablet_manager.cc:1939] T a271f83fa2a44020b676aa92f5c2e907 P 17f3613e4baa429a812f996d5d8219e7: Deleting consensus metadata
I20260807 18:13:58.191661 1490 catalog_manager.cc:4841] TS 17f3613e4baa429a812f996d5d8219e7 (127.18.154.1:46729): tablet a271f83fa2a44020b676aa92f5c2e907 (table test-table [id=4041e929e3064654af3f8f0a47e255f5]) successfully deleted
I20260807 18:13:58.192052 2221 raft_consensus.cc:2258] T a271f83fa2a44020b676aa92f5c2e907 P 84917a8576f442039a9416004e1fcad7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:13:58.195228 2221 ts_tablet_manager.cc:1905] T a271f83fa2a44020b676aa92f5c2e907 P 84917a8576f442039a9416004e1fcad7: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 07 18:13:58 dist-test-slave-gkw7 krb5kdc[1446](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126435, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.154.1@KRBTEST.COM
I20260807 18:13:58.202653 2223 tablet_replica.cc:330] T a271f83fa2a44020b676aa92f5c2e907 P fed13caab5144cb69467a39c4c11037c: stopping tablet replica
I20260807 18:13:58.203583 2223 raft_consensus.cc:2229] T a271f83fa2a44020b676aa92f5c2e907 P fed13caab5144cb69467a39c4c11037c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:13:58.204234 2223 raft_consensus.cc:2258] T a271f83fa2a44020b676aa92f5c2e907 P fed13caab5144cb69467a39c4c11037c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:13:58.206708 2221 ts_tablet_manager.cc:1918] T a271f83fa2a44020b676aa92f5c2e907 P 84917a8576f442039a9416004e1fcad7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:13:58.206892 2223 ts_tablet_manager.cc:1905] T a271f83fa2a44020b676aa92f5c2e907 P fed13caab5144cb69467a39c4c11037c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:13:58.206943 2221 log.cc:1196] T a271f83fa2a44020b676aa92f5c2e907 P 84917a8576f442039a9416004e1fcad7: Deleting WAL directory at /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126328808105-19048-0/minicluster-data/ts-2/wal/wals/a271f83fa2a44020b676aa92f5c2e907
I20260807 18:13:58.207533 2221 ts_tablet_manager.cc:1939] T a271f83fa2a44020b676aa92f5c2e907 P 84917a8576f442039a9416004e1fcad7: Deleting consensus metadata
I20260807 18:13:58.208704 2182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:58.156663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42348 (local address 127.18.154.1:46729)
0807 18:13:58.156907 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:58.156913 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:58.180995 (+ 24082us) server_negotiation.cc:407] Connection header received
0807 18:13:58.181358 (+ 363us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:58.181368 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:58.181510 (+ 142us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:58.181675 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:58.186064 (+ 4389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:58.187797 (+ 1733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:58.191912 (+ 4115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:58.192891 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:58.193016 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:58.200563 (+ 7547us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:13:58.200589 (+ 26us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:13:58.200615 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:13:58.200689 (+ 74us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:13:58.204636 (+ 3947us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:58.205645 (+ 1009us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:58.205655 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:58.205659 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:58.205857 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:58.206378 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:58.206383 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:58.206386 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:58.206859 (+ 473us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:13:58.207222 (+ 363us) server_negotiation.cc:1035] Waiting for connection context
0807 18:13:58.207901 (+ 679us) server_negotiation.cc:299] Negotiation successful
0807 18:13:58.208172 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260807 18:13:58.209713 1491 catalog_manager.cc:4841] TS 84917a8576f442039a9416004e1fcad7 (127.18.154.3:41403): tablet a271f83fa2a44020b676aa92f5c2e907 (table test-table [id=4041e929e3064654af3f8f0a47e255f5]) successfully deleted
I20260807 18:13:58.210739 1761 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:13:58.215466 2223 ts_tablet_manager.cc:1918] T a271f83fa2a44020b676aa92f5c2e907 P fed13caab5144cb69467a39c4c11037c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:13:58.215636 2223 log.cc:1196] T a271f83fa2a44020b676aa92f5c2e907 P fed13caab5144cb69467a39c4c11037c: Deleting WAL directory at /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786126328808105-19048-0/minicluster-data/ts-1/wal/wals/a271f83fa2a44020b676aa92f5c2e907
I20260807 18:13:58.216189 2223 ts_tablet_manager.cc:1939] T a271f83fa2a44020b676aa92f5c2e907 P fed13caab5144cb69467a39c4c11037c: Deleting consensus metadata
I20260807 18:13:58.217778 1491 catalog_manager.cc:4841] TS fed13caab5144cb69467a39c4c11037c (127.18.154.2:34637): tablet a271f83fa2a44020b676aa92f5c2e907 (table test-table [id=4041e929e3064654af3f8f0a47e255f5]) successfully deleted
I20260807 18:13:58.313014 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 1588
I20260807 18:13:58.342658 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 1774
Aug 07 18:13:58 dist-test-slave-gkw7 krb5kdc[1446](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126435, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.154.2@KRBTEST.COM
I20260807 18:13:58.358661 2180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:58.343493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40468 (local address 127.18.154.2:34637)
0807 18:13:58.343718 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:58.343724 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:58.343745 (+ 21us) server_negotiation.cc:407] Connection header received
0807 18:13:58.343927 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:58.343933 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:58.344061 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:58.344173 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:58.345589 (+ 1416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:58.346344 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:58.347765 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:58.348174 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:58.348317 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:58.352941 (+ 4624us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:13:58.352958 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:13:58.352976 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:13:58.353020 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:13:58.355742 (+ 2722us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:58.356468 (+ 726us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:58.356477 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:58.356479 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:58.356613 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:58.357018 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:58.357023 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:58.357026 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:58.357419 (+ 393us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:13:58.357782 (+ 363us) server_negotiation.cc:1035] Waiting for connection context
0807 18:13:58.358063 (+ 281us) server_negotiation.cc:299] Negotiation successful
0807 18:13:58.358304 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260807 18:13:58.359401 1951 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:13:58.451280 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 1774
I20260807 18:13:58.474308 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 1967
Aug 07 18:13:58 dist-test-slave-gkw7 krb5kdc[1446](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126435, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.154.3@KRBTEST.COM
I20260807 18:13:58.491277 2184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:13:58.475277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56452 (local address 127.18.154.3:41403)
0807 18:13:58.475474 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0807 18:13:58.475482 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:13:58.475576 (+ 94us) server_negotiation.cc:407] Connection header received
0807 18:13:58.475800 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:13:58.475810 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:13:58.475890 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:13:58.476001 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:13:58.477376 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:58.478194 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:58.479608 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:13:58.480018 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:13:58.480164 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:13:58.485230 (+ 5066us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:13:58.485248 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:13:58.485267 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:13:58.485313 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:13:58.488338 (+ 3025us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:58.489215 (+ 877us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:58.489224 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:58.489226 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:58.489386 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:13:58.489834 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:13:58.489838 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:13:58.489840 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:13:58.490305 (+ 465us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:13:58.490536 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0807 18:13:58.490821 (+ 285us) server_negotiation.cc:299] Negotiation successful
0807 18:13:58.490994 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260807 18:13:58.491992 2155 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:13:58.581890 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 1967
I20260807 18:13:58.606334 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 1464
I20260807 18:13:58.607583 1547 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:13:58.718482 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 1464
2026-08-07T18:13:58Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (4918 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126328808105-19048-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:13:58 dist-test-slave-gkw7 krb5kdc[2234](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:13:58 dist-test-slave-gkw7 krb5kdc[2234](info): set up 2 sockets
Aug 07 18:13:58 dist-test-slave-gkw7 krb5kdc[2234](info): commencing operation
krb5kdc: starting...
W20260807 18:14:00.029256 19048 mini_kdc.cc:121] Time spent starting KDC: real 1.260s user 0.003s sys 0.035s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126328808105-19048-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126328808105-19048-0/krb5kdc/test-user.keytab.
Aug 07 18:14:00 dist-test-slave-gkw7 krb5kdc[2234](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126440, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:14:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:14:00Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.18.154.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.18.154.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.18.154.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126328808105-19048-0/krb5kdc/rangeradmin_127.18.154.20@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.18.154.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126328808105-19048-0/krb5kdc/rangeradmin_127.18.154.20@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.18.154.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.18.154.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.18.154.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126328808105-19048-0/krb5kdc/rangerlookup_127.18.154.20@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.18.154.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126328808105-19048-0/krb5kdc/rangerlookup_127.18.154.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.18.154.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.18.154.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.18.154.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126328808105-19048-0/krb5kdc/HTTP_127.18.154.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.18.154.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126328808105-19048-0/krb5kdc/HTTP_127.18.154.20@KRBTEST.COM.keytab.
I20260807 18:14:00.469491 19048 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126328808105-19048-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskTVw2hL/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126328808105-19048-0/minicluster-data/postgres -l logfile start
2026-08-07 18:14:03.270 UTC [2280] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-07 18:14:03.270 UTC [2280] LOG: listening on IPv4 address "127.18.154.20", port 38841
2026-08-07 18:14:03.274 UTC [2280] LOG: listening on Unix socket "/tmp/.s.PGSQL.38841"
2026-08-07 18:14:03.304 UTC [2283] LOG: database system was shut down at 2026-08-07 18:14:01 UTC
2026-08-07 18:14:03.308 UTC [2280] LOG: database system is ready to accept connections
I20260807 18:14:04.467435 19048 mini_postgres.cc:96] Postgres bound to 38841
2026-08-07 18:14:04.492 UTC [2294] FATAL: database "slave" does not exist
127.18.154.20:38841 - accepting connections
I20260807 18:14:04.493891 19048 mini_ranger.cc:161] Starting Ranger...
I20260807 18:14:04.528409 19048 mini_ranger.cc:84] Created miniranger Postgres user
I20260807 18:14:05.077490 19048 mini_ranger.cc:87] Created ranger Postgres database
I20260807 18:14:05.077770 19048 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126328808105-19048-0/minicluster-data/ranger-admin
2026-08-07 18:14:05,482 [I] DB FLAVOR :POSTGRES
2026-08-07 18:14:05,483 [I] --------- Verifying Ranger DB connection ---------
2026-08-07 18:14:05,483 [I] Checking connection..
2026-08-07 18:14:05,483 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTVw2hL/build/asan/bin/postgresql.jar:/tmp/dist-test-taskTVw2hL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.154.20:38841/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-07 18:14:05,757 [I] Checking connection passed.
2026-08-07 18:14:05,757 [I] --------- Verifying version history table ---------
2026-08-07 18:14:05,757 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTVw2hL/build/asan/bin/postgresql.jar:/tmp/dist-test-taskTVw2hL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.154.20:38841/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 18:14:06,017 [I] Table x_db_version_h does not exist in database ranger
2026-08-07 18:14:06,018 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTVw2hL/build/asan/bin/postgresql.jar:/tmp/dist-test-taskTVw2hL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.154.20:38841/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 18:14:06,272 [I] Table x_db_version_h does not exist in database ranger
2026-08-07 18:14:06,273 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-07 18:14:06,273 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTVw2hL/build/asan/bin/postgresql.jar:/tmp/dist-test-taskTVw2hL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.154.20:38841/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskTVw2hL/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-07 18:14:06.524 UTC [2383] WARNING: there is no transaction in progress
2026-08-07 18:14:06,533 [I] create_dbversion_catalog.sql file imported successfully
2026-08-07 18:14:06,533 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTVw2hL/build/asan/bin/postgresql.jar:/tmp/dist-test-taskTVw2hL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.154.20:38841/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 18:14:06,781 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-07 18:14:06,781 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-07 18:14:06,782 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTVw2hL/build/asan/bin/postgresql.jar:/tmp/dist-test-taskTVw2hL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.154.20:38841/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-07 18:14:07,055 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTVw2hL/build/asan/bin/postgresql.jar:/tmp/dist-test-taskTVw2hL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.154.20:38841/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-07 18:14:07,317 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTVw2hL/build/asan/bin/postgresql.jar:/tmp/dist-test-taskTVw2hL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.154.20:38841/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-gkw7.c.gcp-upstream.internal','N') ;"
2026-08-07 18:14:07,581 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTVw2hL/build/asan/bin/postgresql.jar:/tmp/dist-test-taskTVw2hL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.154.20:38841/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-07 18:14:07,838 [I] Table x_portal_user does not exist in database ranger
2026-08-07 18:14:07,838 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTVw2hL/build/asan/bin/postgresql.jar:/tmp/dist-test-taskTVw2hL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.154.20:38841/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-07 18:14:08,089 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-07 18:14:08,089 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTVw2hL/build/asan/bin/postgresql.jar:/tmp/dist-test-taskTVw2hL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.154.20:38841/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-07 18:14:08,340 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTVw2hL/build/asan/bin/postgresql.jar:/tmp/dist-test-taskTVw2hL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.154.20:38841/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-07 18:14:08,613 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-07 18:14:08,614 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTVw2hL/build/asan/bin/postgresql.jar:/tmp/dist-test-taskTVw2hL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.154.20:38841/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskTVw2hL/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-07 18:14:09.658 UTC [2539] WARNING: there is no transaction in progress
2026-08-07 18:14:09.675 UTC [2539] WARNING: there is no transaction in progress
2026-08-07 18:14:09.692 UTC [2539] WARNING: there is no transaction in progress
2026-08-07 18:14:09.709 UTC [2539] WARNING: there is no transaction in progress
2026-08-07 18:14:09.727 UTC [2539] WARNING: there is no transaction in progress
2026-08-07 18:14:09.746 UTC [2539] WARNING: there is no transaction in progress
2026-08-07 18:14:09.812 UTC [2539] WARNING: there is no transaction in progress
2026-08-07 18:14:09.819 UTC [2539] WARNING: there is no transaction in progress
2026-08-07 18:14:09.835 UTC [2539] WARNING: there is no transaction in progress
2026-08-07 18:14:09.846 UTC [2539] WARNING: there is no transaction in progress
2026-08-07 18:14:09.856 UTC [2539] WARNING: there is no transaction in progress
2026-08-07 18:14:09.870 UTC [2539] WARNING: there is no transaction in progress
2026-08-07 18:14:09.880 UTC [2539] WARNING: there is no transaction in progress
2026-08-07 18:14:09.886 UTC [2539] WARNING: there is no transaction in progress
2026-08-07 18:14:09.893 UTC [2539] WARNING: there is no transaction in progress
2026-08-07 18:14:09.900 UTC [2539] WARNING: there is no transaction in progress
2026-08-07 18:14:10,606 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-07 18:14:10,607 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTVw2hL/build/asan/bin/postgresql.jar:/tmp/dist-test-taskTVw2hL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.154.20:38841/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-07 18:14:10,864 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-07 18:14:10,864 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTVw2hL/build/asan/bin/postgresql.jar:/tmp/dist-test-taskTVw2hL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.154.20:38841/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-07 18:14:11,147 [I] Table x_portal_user already exists in database 'ranger'
2026-08-07 18:14:11,148 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTVw2hL/build/asan/bin/postgresql.jar:/tmp/dist-test-taskTVw2hL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.154.20:38841/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-07 18:14:11,423 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-07 18:14:11,423 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTVw2hL/build/asan/bin/postgresql.jar:/tmp/dist-test-taskTVw2hL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.154.20:38841/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-07 18:14:11,689 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTVw2hL/build/asan/bin/postgresql.jar:/tmp/dist-test-taskTVw2hL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.154.20:38841/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-07 18:14:11,981 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTVw2hL/build/asan/bin/postgresql.jar:/tmp/dist-test-taskTVw2hL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.154.20:38841/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-gkw7.c.gcp-upstream.internal';"
2026-08-07 18:14:12,223 [I] CORE_DB_SCHEMA import status has been updated
2026-08-07 18:14:12,224 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTVw2hL/build/asan/bin/postgresql.jar:/tmp/dist-test-taskTVw2hL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.154.20:38841/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-07 18:14:12,488 [I] DB_PATCHES have already been applied
I20260807 18:14:12.495018 19048 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126328808105-19048-0/minicluster-data/ranger-admin:/tmp/dist-test-taskTVw2hL/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskTVw2hL/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskTVw2hL/build/asan/bin/postgresql.jar:/tmp/dist-test-taskTVw2hL/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260807 18:14:12.495134 19048 mini_ranger.cc:193] Using host: 127.18.154.20
I20260807 18:14:12.517391 19048 mini_ranger.cc:227] Ranger admin URL: http://127.18.154.20:59465
Aug 07, 2026 6:14:12 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 07, 2026 6:14:12 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 07, 2026 6:14:13 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126328808105-19048-0/minicluster-data/ranger-admin/ews/webapp
Aug 07, 2026 6:14:13 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126328808105-19048-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 07, 2026 6:14:13 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126328808105-19048-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 07, 2026 6:14:13 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126328808105-19048-0/minicluster-data/ranger-admin/ews/webapp].
Aug 07, 2026 6:14:13 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260807 18:14:13.960314 19048 mini_ranger.cc:160] Time spent starting Ranger: real 9.466s user 0.000s sys 0.066s
Aug 07, 2026 6:14:34 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 07, 2026 6:14:34 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Aug 07, 2026 6:14:34 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 07, 2026 6:14:34 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 07, 2026 6:14:34 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 07, 2026 6:14:34 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 07, 2026 6:14:34 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 07, 2026 6:14:34 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 07, 2026 6:14:34 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 07, 2026 6:14:34 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 07, 2026 6:14:34 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 07, 2026 6:14:34 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 07, 2026 6:14:34 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 07, 2026 6:14:34 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 07, 2026 6:14:34 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 07, 2026 6:14:34 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 07, 2026 6:14:34 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 07, 2026 6:14:35 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260807 18:14:36.101490 19048 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.18.154.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.18.154.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.18.154.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126328808105-19048-0/krb5kdc/rangerkms_127.18.154.20@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.18.154.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126328808105-19048-0/krb5kdc/rangerkms_127.18.154.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.18.154.20@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.18.154.20@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.18.154.20@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126328808105-19048-0/krb5kdc/HTTP_127.18.154.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.18.154.20@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126328808105-19048-0/krb5kdc/HTTP_127.18.154.20@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 07 18:14:36 dist-test-slave-gkw7 krb5kdc[2234](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126476, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260807 18:14:36.328353 19048 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260807 18:14:36.355692 19048 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260807 18:14:37.299365 19048 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260807 18:14:37.299516 19048 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126328808105-19048-0/minicluster-data/ranger-kms
I20260807 18:14:37.299681 19048 mini_ranger_kms.cc:227] Using postgres at 127.18.154.20:38841
2026-08-07 18:14:37,954 [I] DB FLAVOR :POSTGRES
2026-08-07 18:14:37,955 [I] --------- Verifying Ranger DB connection ---------
2026-08-07 18:14:37,955 [I] Checking connection
2026-08-07 18:14:37,955 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTVw2hL/build/asan/bin/postgresql.jar:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126328808105-19048-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.154.20:38841/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-07 18:14:38,213 [I] connection success
2026-08-07 18:14:38,214 [I] --------- Verifying Ranger DB tables ---------
2026-08-07 18:14:38,214 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-07 18:14:38,214 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTVw2hL/build/asan/bin/postgresql.jar:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126328808105-19048-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.154.20:38841/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-07 18:14:38,468 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-07 18:14:38,468 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-07 18:14:38,468 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-07 18:14:38,468 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTVw2hL/build/asan/bin/postgresql.jar:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126328808105-19048-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.154.20:38841/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126328808105-19048-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-07 18:14:38,750 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260807 18:14:39.258739 19048 mini_ranger_kms.cc:310] Created kms service
I20260807 18:14:39.494144 19048 mini_ranger_kms.cc:326] Created kudu user
I20260807 18:14:39.541360 19048 mini_ranger_kms.cc:343] Created rangerkms user
Aug 07 18:14:39 dist-test-slave-gkw7 krb5kdc[2234](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126476, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.18.154.20@KRBTEST.COM
I20260807 18:14:39.958052 19048 mini_ranger_kms.cc:384] Added ranger policy
I20260807 18:14:39.959442 19048 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126328808105-19048-0/minicluster-data/ranger-kms:/tmp/dist-test-taskTVw2hL/build/asan/bin/postgresql.jar:/tmp/dist-test-taskTVw2hL/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskTVw2hL/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260807 18:14:39.959522 19048 mini_ranger_kms.cc:240] Using host: 127.18.154.20
I20260807 18:14:39.988638 19048 mini_ranger_kms.cc:276] Ranger KMS PID: 2804
I20260807 18:14:39.988852 19048 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.18.154.20:59295
Aug 07, 2026 6:14:41 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 07, 2026 6:14:41 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 07, 2026 6:14:41 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126328808105-19048-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 07, 2026 6:14:41 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126328808105-19048-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 07, 2026 6:14:41 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126328808105-19048-0/minicluster-data/ranger-kms/ews/webapp].
Aug 07, 2026 6:14:41 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 07 18:14:42 dist-test-slave-gkw7 krb5kdc[2234](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786126482, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.18.154.20@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:14:42.744390 19048 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.416s user 0.152s sys 0.553s
I20260807 18:14:42.744740 19048 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260807 18:14:42.744993 19048 mini_ranger_kms.cc:401] 127.18.154.20:59295/kms/v1/keys
Aug 07 18:14:42 dist-test-slave-gkw7 krb5kdc[2234](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126482, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.18.154.20@KRBTEST.COM for HTTP/127.18.154.20@KRBTEST.COM
Aug 07 18:14:43 dist-test-slave-gkw7 krb5kdc[2234](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126482, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.18.154.20@KRBTEST.COM for HTTP/127.18.154.20@KRBTEST.COM
Aug 07 18:14:45 dist-test-slave-gkw7 krb5kdc[2234](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126485, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.18.154.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.154.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.154.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.62.keytab.
Entry for principal kudu/127.18.154.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.62.keytab.
I20260807 18:14:45.934418 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126328808105-19048-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126328808105-19048-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126328808105-19048-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.154.62:37391
--webserver_interface=127.18.154.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.62.keytab
--principal=kudu/127.18.154.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.154.20:39619
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.154.62:37391
--ranger_config_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126328808105-19048-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.154.20:59295/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126328808105-19048-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126328808105-19048-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126328808105-19048-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:14:46.336895 2875 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:14:46.337235 2875 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:14:46.337424 2875 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:14:46.345360 2875 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:14:46.345471 2875 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:14:46.345603 2875 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:14:46.345705 2875 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:14:46.345810 2875 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:14:46.356824 2875 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.154.20:39619
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126328808105-19048-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126328808105-19048-0/minicluster-data/master-0/wal
--ranger_kms_url=127.18.154.20:59295/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.18.154.62:37391
--ranger_config_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126328808105-19048-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.62.keytab
--principal=kudu/127.18.154.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.154.62:37391
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.154.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126328808105-19048-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:14:46.358320 2875 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:14:46.359735 2875 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:14:46.373420 2882 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:14:46.373557 2883 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:14:46.373819 2885 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:14:46.374402 2875 server_base.cc:1034] running on GCE node
I20260807 18:14:46.375252 2875 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:14:46.376868 2875 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:14:46.378172 2875 hybrid_clock.cc:648] HybridClock initialized: now 1786126486378044 us; error 54 us; skew 500 ppm
I20260807 18:14:46.378616 2875 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:14:46 dist-test-slave-gkw7 krb5kdc[2234](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126486, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:14:46.382354 2875 init.cc:377] Logged in from keytab as kudu/127.18.154.62@KRBTEST.COM (short username kudu)
I20260807 18:14:46.383948 2875 webserver.cc:460] Webserver started at http://127.18.154.62:45979/ using document root <none> and password file <none>
I20260807 18:14:46.384650 2875 fs_manager.cc:359] Metadata directory not provided
I20260807 18:14:46.384809 2875 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:14:46.385152 2875 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 18:14:46 dist-test-slave-gkw7 krb5kdc[2234](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126486, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.62@KRBTEST.COM for HTTP/127.18.154.20@KRBTEST.COM
I20260807 18:14:46.448632 2875 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126328808105-19048-0/minicluster-data/master-0/data/instance:
uuid: "c6ff551ba7dc4b1995b065b1459f7b02"
format_stamp: "Formatted at 2026-08-07 18:14:46 on dist-test-slave-gkw7"
server_key: "b66bf280456ccf1fc7ff06566529bb43"
server_key_iv: "9e3cbf7fa38e5ee215ebbf0c784d9a22"
server_key_version: "kuduclusterkey@0"
I20260807 18:14:46.449884 2875 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126328808105-19048-0/minicluster-data/master-0/wal/instance:
uuid: "c6ff551ba7dc4b1995b065b1459f7b02"
format_stamp: "Formatted at 2026-08-07 18:14:46 on dist-test-slave-gkw7"
server_key: "b66bf280456ccf1fc7ff06566529bb43"
server_key_iv: "9e3cbf7fa38e5ee215ebbf0c784d9a22"
server_key_version: "kuduclusterkey@0"
I20260807 18:14:46.458395 2875 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.006s sys 0.001s
I20260807 18:14:46.480038 2899 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:14:46.481961 2875 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 18:14:46.482165 2875 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126328808105-19048-0/minicluster-data/master-0/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126328808105-19048-0/minicluster-data/master-0/wal
uuid: "c6ff551ba7dc4b1995b065b1459f7b02"
format_stamp: "Formatted at 2026-08-07 18:14:46 on dist-test-slave-gkw7"
server_key: "b66bf280456ccf1fc7ff06566529bb43"
server_key_iv: "9e3cbf7fa38e5ee215ebbf0c784d9a22"
server_key_version: "kuduclusterkey@0"
I20260807 18:14:46.482463 2875 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126328808105-19048-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126328808105-19048-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126328808105-19048-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:14:46.517724 2875 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:14:46.522290 2875 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:14:46.522606 2875 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:14:46.548532 2875 rpc_server.cc:307] RPC server started. Bound to: 127.18.154.62:37391
I20260807 18:14:46.548568 2967 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.154.62:37391 every 8 connection(s)
I20260807 18:14:46.550403 2875 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
I20260807 18:14:46.554147 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 2875
I20260807 18:14:46.554355 19048 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126328808105-19048-0/minicluster-data/master-0/wal/instance
I20260807 18:14:46.555258 19048 external_mini_cluster.cc:1428] Setting key 9c41d8aa6f46e535edd52c7c4f039169
I20260807 18:14:46.555501 2968 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:14:46.571580 2968 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c6ff551ba7dc4b1995b065b1459f7b02: Bootstrap starting.
Aug 07 18:14:46 dist-test-slave-gkw7 krb5kdc[2234](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126485, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.154.62@KRBTEST.COM
I20260807 18:14:46.576125 2968 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c6ff551ba7dc4b1995b065b1459f7b02: Neither blocks nor log segments found. Creating new log.
I20260807 18:14:46.577638 2968 log.cc:824] T 00000000000000000000000000000000 P c6ff551ba7dc4b1995b065b1459f7b02: Log is configured to *not* fsync() on all Append() calls
I20260807 18:14:46.581297 2968 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c6ff551ba7dc4b1995b065b1459f7b02: No bootstrap required, opened a new log
I20260807 18:14:46.582568 2971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:46.557684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56368 (local address 127.18.154.62:37391)
0807 18:14:46.558621 (+ 937us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:46.558638 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:46.558685 (+ 47us) server_negotiation.cc:407] Connection header received
0807 18:14:46.560833 (+ 2148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:46.560859 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:46.561333 (+ 474us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:46.561902 (+ 569us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:14:46.563703 (+ 1801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:46.565385 (+ 1682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:46.567226 (+ 1841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:46.567968 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:46.568112 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:46.572901 (+ 4789us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:14:46.572913 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:14:46.572935 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:14:46.572986 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:14:46.577747 (+ 4761us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:46.578576 (+ 829us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:46.578583 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:46.578594 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:46.578736 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:46.579238 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:46.579242 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:46.579245 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:46.579917 (+ 672us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:14:46.580244 (+ 327us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:46.580624 (+ 380us) server_negotiation.cc:299] Negotiation successful
0807 18:14:46.581026 (+ 402us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":696,"thread_start_us":466,"threads_started":1}
I20260807 18:14:46.590902 2968 raft_consensus.cc:348] T 00000000000000000000000000000000 P c6ff551ba7dc4b1995b065b1459f7b02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6ff551ba7dc4b1995b065b1459f7b02" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 37391 } }
I20260807 18:14:46.591248 2968 raft_consensus.cc:374] T 00000000000000000000000000000000 P c6ff551ba7dc4b1995b065b1459f7b02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:14:46.591338 2968 raft_consensus.cc:729] T 00000000000000000000000000000000 P c6ff551ba7dc4b1995b065b1459f7b02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c6ff551ba7dc4b1995b065b1459f7b02, State: Initialized, Role: FOLLOWER
I20260807 18:14:46.591903 2968 consensus_queue.cc:260] T 00000000000000000000000000000000 P c6ff551ba7dc4b1995b065b1459f7b02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6ff551ba7dc4b1995b065b1459f7b02" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 37391 } }
I20260807 18:14:46.592135 2968 raft_consensus.cc:388] T 00000000000000000000000000000000 P c6ff551ba7dc4b1995b065b1459f7b02 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:14:46.592243 2968 raft_consensus.cc:482] T 00000000000000000000000000000000 P c6ff551ba7dc4b1995b065b1459f7b02 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:14:46.592370 2968 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c6ff551ba7dc4b1995b065b1459f7b02 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:14:46.594614 2968 raft_consensus.cc:504] T 00000000000000000000000000000000 P c6ff551ba7dc4b1995b065b1459f7b02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6ff551ba7dc4b1995b065b1459f7b02" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 37391 } }
I20260807 18:14:46.595131 2968 leader_election.cc:302] T 00000000000000000000000000000000 P c6ff551ba7dc4b1995b065b1459f7b02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c6ff551ba7dc4b1995b065b1459f7b02; no voters:
I20260807 18:14:46.596124 2968 leader_election.cc:288] T 00000000000000000000000000000000 P c6ff551ba7dc4b1995b065b1459f7b02 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:14:46.596290 2974 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c6ff551ba7dc4b1995b065b1459f7b02 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:14:46.597155 2974 raft_consensus.cc:686] T 00000000000000000000000000000000 P c6ff551ba7dc4b1995b065b1459f7b02 [term 1 LEADER]: Becoming Leader. State: Replica: c6ff551ba7dc4b1995b065b1459f7b02, State: Running, Role: LEADER
I20260807 18:14:46.597822 2974 consensus_queue.cc:237] T 00000000000000000000000000000000 P c6ff551ba7dc4b1995b065b1459f7b02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6ff551ba7dc4b1995b065b1459f7b02" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 37391 } }
I20260807 18:14:46.598605 2968 sys_catalog.cc:564] T 00000000000000000000000000000000 P c6ff551ba7dc4b1995b065b1459f7b02 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:14:46.601907 2977 sys_catalog.cc:455] T 00000000000000000000000000000000 P c6ff551ba7dc4b1995b065b1459f7b02 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c6ff551ba7dc4b1995b065b1459f7b02. Latest consensus state: current_term: 1 leader_uuid: "c6ff551ba7dc4b1995b065b1459f7b02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6ff551ba7dc4b1995b065b1459f7b02" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 37391 } } }
I20260807 18:14:46.601325 2975 sys_catalog.cc:455] T 00000000000000000000000000000000 P c6ff551ba7dc4b1995b065b1459f7b02 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c6ff551ba7dc4b1995b065b1459f7b02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6ff551ba7dc4b1995b065b1459f7b02" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 37391 } } }
I20260807 18:14:46.602342 2977 sys_catalog.cc:458] T 00000000000000000000000000000000 P c6ff551ba7dc4b1995b065b1459f7b02 [sys.catalog]: This master's current role is: LEADER
I20260807 18:14:46.602444 2975 sys_catalog.cc:458] T 00000000000000000000000000000000 P c6ff551ba7dc4b1995b065b1459f7b02 [sys.catalog]: This master's current role is: LEADER
I20260807 18:14:46.606863 2987 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:14:46.613088 2987 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:14:46.614034 2968 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126328808105-19048-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260807 18:14:46.636966 2987 catalog_manager.cc:1347] Generated new cluster ID: 9e0792d74afd4bb8ace09b2991da28ae
I20260807 18:14:46.637173 2987 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:14:46.654806 2987 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:14:46.656924 2987 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:14:46.664961 2987 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c6ff551ba7dc4b1995b065b1459f7b02: Generated new TSK 0
I20260807 18:14:46.666057 2987 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.154.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.154.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.154.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.1.keytab.
Entry for principal kudu/127.18.154.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.1.keytab.
I20260807 18:14:47.560819 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126328808105-19048-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126328808105-19048-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126328808105-19048-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126328808105-19048-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.154.20:59295/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.154.1:0
--local_ip_for_outbound_sockets=127.18.154.1
--webserver_interface=127.18.154.1
--webserver_port=0
--tserver_master_addrs=127.18.154.62:37391
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.1.keytab
--principal=kudu/127.18.154.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.154.20:39619
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126328808105-19048-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126328808105-19048-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126328808105-19048-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 18:14:47 dist-test-slave-gkw7 krb5kdc[2234](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786126487, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260807 18:14:48.081471 3034 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:14:48.081943 3034 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:14:48.082139 3034 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:14:48.093140 3034 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:14:48.093438 3034 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:14:48.093729 3034 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.154.1
I20260807 18:14:48.108686 3034 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.154.20:39619
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126328808105-19048-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126328808105-19048-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.18.154.20:59295/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.1.keytab
--principal=kudu/127.18.154.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.154.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126328808105-19048-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.154.1
--webserver_port=0
--tserver_master_addrs=127.18.154.62:37391
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.154.1
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126328808105-19048-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:14:48.110865 3034 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:14:48.113300 3034 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:14:48.137326 3046 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:14:48.138399 3049 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:14:48.148522 3051 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:14:48.149323 3034 server_base.cc:1034] running on GCE node
I20260807 18:14:48.150156 3034 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:14:48.151331 3034 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:14:48.152616 3034 hybrid_clock.cc:648] HybridClock initialized: now 1786126488152549 us; error 69 us; skew 500 ppm
I20260807 18:14:48.153004 3034 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:14:48 dist-test-slave-gkw7 krb5kdc[2234](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126488, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:14:48.157078 3034 init.cc:377] Logged in from keytab as kudu/127.18.154.1@KRBTEST.COM (short username kudu)
I20260807 18:14:48.163090 3034 webserver.cc:460] Webserver started at http://127.18.154.1:39389/ using document root <none> and password file <none>
I20260807 18:14:48.163978 3034 fs_manager.cc:359] Metadata directory not provided
I20260807 18:14:48.164150 3034 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:14:48.164601 3034 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 18:14:48 dist-test-slave-gkw7 krb5kdc[2234](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126488, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.1@KRBTEST.COM for HTTP/127.18.154.20@KRBTEST.COM
I20260807 18:14:48.192981 3034 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126328808105-19048-0/minicluster-data/ts-0/data/instance:
uuid: "d9cb0ea1ee744b7bbc3e975c7c817b22"
format_stamp: "Formatted at 2026-08-07 18:14:48 on dist-test-slave-gkw7"
server_key: "c39ef2dc28c6d025d63172c0db1c4767"
server_key_iv: "30f07169d88918e825f1d3cb7116fb38"
server_key_version: "kuduclusterkey@0"
I20260807 18:14:48.194434 3034 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126328808105-19048-0/minicluster-data/ts-0/wal/instance:
uuid: "d9cb0ea1ee744b7bbc3e975c7c817b22"
format_stamp: "Formatted at 2026-08-07 18:14:48 on dist-test-slave-gkw7"
server_key: "c39ef2dc28c6d025d63172c0db1c4767"
server_key_iv: "30f07169d88918e825f1d3cb7116fb38"
server_key_version: "kuduclusterkey@0"
I20260807 18:14:48.206413 3034 fs_manager.cc:691] Time spent creating directory manager: real 0.011s user 0.005s sys 0.005s
I20260807 18:14:48.236119 3059 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:14:48.239622 3034 fs_manager.cc:725] Time spent opening block manager: real 0.011s user 0.003s sys 0.002s
I20260807 18:14:48.239889 3034 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126328808105-19048-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126328808105-19048-0/minicluster-data/ts-0/wal
uuid: "d9cb0ea1ee744b7bbc3e975c7c817b22"
format_stamp: "Formatted at 2026-08-07 18:14:48 on dist-test-slave-gkw7"
server_key: "c39ef2dc28c6d025d63172c0db1c4767"
server_key_iv: "30f07169d88918e825f1d3cb7116fb38"
server_key_version: "kuduclusterkey@0"
I20260807 18:14:48.240293 3034 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126328808105-19048-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126328808105-19048-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126328808105-19048-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:14:48.272195 3034 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:14:48.279911 3034 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:14:48.280666 3034 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:14:48.284650 3034 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:14:48.287726 3034 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:14:48.287848 3034 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:14:48.288008 3034 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:14:48.288089 3034 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 18:14:48 dist-test-slave-gkw7 krb5kdc[2234](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126487, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.62@KRBTEST.COM for HTTP/127.18.154.20@KRBTEST.COM
I20260807 18:14:48.499867 3034 rpc_server.cc:307] RPC server started. Bound to: 127.18.154.1:44579
I20260807 18:14:48.499938 3227 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.154.1:44579 every 8 connection(s)
I20260807 18:14:48.502159 3034 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126328808105-19048-0/minicluster-data/ts-0/data/info.pb
I20260807 18:14:48.509904 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 3034
I20260807 18:14:48.510097 19048 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126328808105-19048-0/minicluster-data/ts-0/wal/instance
I20260807 18:14:48.510931 19048 external_mini_cluster.cc:1428] Setting key e9b4d8f602ecfa0ffc1b58eaf1366d4d
Aug 07 18:14:48 dist-test-slave-gkw7 krb5kdc[2234](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126488, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.1@KRBTEST.COM for kudu/127.18.154.62@KRBTEST.COM
I20260807 18:14:48.552714 3234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:48.507472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.1:34517 (local address 127.18.154.62:37391)
0807 18:14:48.526133 (+ 18661us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:48.526140 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:48.527789 (+ 1649us) server_negotiation.cc:407] Connection header received
0807 18:14:48.530257 (+ 2468us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:48.530263 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:48.530375 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:48.530508 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:14:48.532949 (+ 2441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:48.534320 (+ 1371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:48.537330 (+ 3010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:48.537916 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:48.538049 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:48.543102 (+ 5053us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:14:48.543110 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:14:48.543114 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:14:48.543158 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:14:48.547685 (+ 4527us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:48.548901 (+ 1216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:48.548909 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:48.548914 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:48.549171 (+ 257us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:48.549998 (+ 827us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:48.550005 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:48.550008 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:48.550530 (+ 522us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:14:48.550673 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:48.551953 (+ 1280us) server_negotiation.cc:299] Negotiation successful
0807 18:14:48.552233 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18489,"thread_start_us":18369,"threads_started":1}
I20260807 18:14:48.556200 3232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:48.506842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:37391 (local address 127.18.154.1:34517)
0807 18:14:48.526865 (+ 20023us) negotiation.cc:107] Waiting for socket to connect
0807 18:14:48.526925 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0807 18:14:48.529876 (+ 2951us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:14:48.530852 (+ 976us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:14:48.530868 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:14:48.531631 (+ 763us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:14:48.532707 (+ 1076us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:48.532742 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:48.534614 (+ 1872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:48.534622 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:48.537142 (+ 2520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:48.537173 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:48.538217 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:48.538229 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:48.538819 (+ 590us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:48.539744 (+ 925us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:14:48.539798 (+ 54us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:14:48.542799 (+ 3001us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:14:48.547906 (+ 5107us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:14:48.547919 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:14:48.547941 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:14:48.548723 (+ 782us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:14:48.549402 (+ 679us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:14:48.549410 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:14:48.549414 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:14:48.549815 (+ 401us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:14:48.550800 (+ 985us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:14:48.550810 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:14:48.551428 (+ 618us) client_negotiation.cc:769] Sending connection context
0807 18:14:48.553439 (+ 2011us) client_negotiation.cc:240] Negotiation successful
0807 18:14:48.554113 (+ 674us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19652,"thread_start_us":19411,"threads_started":1}
I20260807 18:14:48.559732 3229 heartbeater.cc:347] Connected to a master server at 127.18.154.62:37391
I20260807 18:14:48.560180 3229 heartbeater.cc:464] Registering TS with master...
I20260807 18:14:48.561331 3229 heartbeater.cc:511] Master 127.18.154.62:37391 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.18.154.2@KRBTEST.COM; defaulting to no policy
I20260807 18:14:48.564841 2920 ts_manager.cc:194] Registered new tserver with Master: d9cb0ea1ee744b7bbc3e975c7c817b22 (127.18.154.1:44579)
I20260807 18:14:48.567612 2920 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.154.1@KRBTEST.COM'} at 127.18.154.1:34517
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.154.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.154.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.2.keytab.
Entry for principal kudu/127.18.154.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.2.keytab.
I20260807 18:14:48.618716 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126328808105-19048-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126328808105-19048-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126328808105-19048-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126328808105-19048-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.154.20:59295/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.154.2:0
--local_ip_for_outbound_sockets=127.18.154.2
--webserver_interface=127.18.154.2
--webserver_port=0
--tserver_master_addrs=127.18.154.62:37391
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.2.keytab
--principal=kudu/127.18.154.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.154.20:39619
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126328808105-19048-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126328808105-19048-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126328808105-19048-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 18:14:48 dist-test-slave-gkw7 krb5kdc[2234](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126487, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.62@KRBTEST.COM for HTTP/127.18.154.20@KRBTEST.COM
W20260807 18:14:49.159936 3238 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:14:49.160255 3238 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:14:49.160413 3238 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:14:49.170032 3238 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:14:49.170151 3238 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:14:49.170343 3238 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.154.2
I20260807 18:14:49.181042 3238 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.154.20:39619
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126328808105-19048-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126328808105-19048-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.18.154.20:59295/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.2.keytab
--principal=kudu/127.18.154.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.154.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126328808105-19048-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.154.2
--webserver_port=0
--tserver_master_addrs=127.18.154.62:37391
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.154.2
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126328808105-19048-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:14:49.182545 3238 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:14:49.184031 3238 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:14:49.200480 3261 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:14:49.200729 3263 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:14:49.200992 3259 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:14:49.201911 3238 server_base.cc:1034] running on GCE node
I20260807 18:14:49.202790 3238 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:14:49.204166 3238 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:14:49.205427 3238 hybrid_clock.cc:648] HybridClock initialized: now 1786126489205365 us; error 87 us; skew 500 ppm
I20260807 18:14:49.205863 3238 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:14:49 dist-test-slave-gkw7 krb5kdc[2234](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126489, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:14:49.210075 3238 init.cc:377] Logged in from keytab as kudu/127.18.154.2@KRBTEST.COM (short username kudu)
I20260807 18:14:49.212178 3238 webserver.cc:460] Webserver started at http://127.18.154.2:46757/ using document root <none> and password file <none>
I20260807 18:14:49.212988 3238 fs_manager.cc:359] Metadata directory not provided
I20260807 18:14:49.213227 3238 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:14:49.213779 3238 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 18:14:49 dist-test-slave-gkw7 krb5kdc[2234](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126489, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.2@KRBTEST.COM for HTTP/127.18.154.20@KRBTEST.COM
I20260807 18:14:49.233515 3238 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126328808105-19048-0/minicluster-data/ts-1/data/instance:
uuid: "bc24572293df41fdb3875f6a7ef35369"
format_stamp: "Formatted at 2026-08-07 18:14:49 on dist-test-slave-gkw7"
server_key: "f0400ca351d93b3eb1316613009f75f6"
server_key_iv: "0801648ff7ddbae824001e6117d82e5a"
server_key_version: "kuduclusterkey@0"
I20260807 18:14:49.234941 3238 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126328808105-19048-0/minicluster-data/ts-1/wal/instance:
uuid: "bc24572293df41fdb3875f6a7ef35369"
format_stamp: "Formatted at 2026-08-07 18:14:49 on dist-test-slave-gkw7"
server_key: "f0400ca351d93b3eb1316613009f75f6"
server_key_iv: "0801648ff7ddbae824001e6117d82e5a"
server_key_version: "kuduclusterkey@0"
I20260807 18:14:49.243124 3238 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.001s
I20260807 18:14:49.262598 3272 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:14:49.264858 3238 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260807 18:14:49.265183 3238 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126328808105-19048-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126328808105-19048-0/minicluster-data/ts-1/wal
uuid: "bc24572293df41fdb3875f6a7ef35369"
format_stamp: "Formatted at 2026-08-07 18:14:49 on dist-test-slave-gkw7"
server_key: "f0400ca351d93b3eb1316613009f75f6"
server_key_iv: "0801648ff7ddbae824001e6117d82e5a"
server_key_version: "kuduclusterkey@0"
I20260807 18:14:49.265745 3238 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126328808105-19048-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126328808105-19048-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126328808105-19048-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:14:49.297123 3238 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:14:49.303330 3238 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:14:49.303822 3238 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:14:49.305347 3238 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:14:49.307614 3238 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:14:49.307780 3238 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260807 18:14:49.307965 3238 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:14:49.308131 3238 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:14:49.384032 3238 rpc_server.cc:307] RPC server started. Bound to: 127.18.154.2:42927
I20260807 18:14:49.384081 3459 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.154.2:42927 every 8 connection(s)
I20260807 18:14:49.385766 3238 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126328808105-19048-0/minicluster-data/ts-1/data/info.pb
I20260807 18:14:49.391059 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 3238
I20260807 18:14:49.391232 19048 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126328808105-19048-0/minicluster-data/ts-1/wal/instance
I20260807 18:14:49.391810 19048 external_mini_cluster.cc:1428] Setting key da6a26897bf311149b1b4c392ab55fdc
Aug 07 18:14:49 dist-test-slave-gkw7 krb5kdc[2234](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126489, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.2@KRBTEST.COM for kudu/127.18.154.62@KRBTEST.COM
I20260807 18:14:49.427556 3464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:49.389847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.2:47587 (local address 127.18.154.62:37391)
0807 18:14:49.395613 (+ 5766us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:49.395621 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:49.404151 (+ 8530us) server_negotiation.cc:407] Connection header received
0807 18:14:49.406982 (+ 2831us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:49.406990 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:49.407113 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:49.407248 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:14:49.409547 (+ 2299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:49.410790 (+ 1243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:49.413480 (+ 2690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:49.414127 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:49.414268 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:49.419366 (+ 5098us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:14:49.419374 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:14:49.419380 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:14:49.419424 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:14:49.423341 (+ 3917us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:49.424458 (+ 1117us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:49.424464 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:49.424467 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:49.424630 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:49.425226 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:49.425232 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:49.425236 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:49.425661 (+ 425us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:14:49.425786 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:49.426887 (+ 1101us) server_negotiation.cc:299] Negotiation successful
0807 18:14:49.427150 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5635,"thread_start_us":8202,"threads_started":1}
I20260807 18:14:49.430294 3465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:49.390279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:37391 (local address 127.18.154.2:47587)
0807 18:14:49.403909 (+ 13630us) negotiation.cc:107] Waiting for socket to connect
0807 18:14:49.403983 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0807 18:14:49.406642 (+ 2659us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:14:49.407550 (+ 908us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:14:49.407563 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:14:49.408244 (+ 681us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:14:49.409337 (+ 1093us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:49.409363 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:49.410935 (+ 1572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:49.410941 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:49.413273 (+ 2332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:49.413322 (+ 49us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:49.414700 (+ 1378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:49.414709 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:49.415007 (+ 298us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:49.415927 (+ 920us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:14:49.415967 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:14:49.419082 (+ 3115us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:14:49.423542 (+ 4460us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:14:49.423552 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:14:49.423574 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:14:49.424290 (+ 716us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:14:49.424791 (+ 501us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:14:49.424795 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:14:49.424799 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:14:49.425072 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:14:49.425896 (+ 824us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:14:49.425905 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:14:49.426404 (+ 499us) client_negotiation.cc:769] Sending connection context
0807 18:14:49.428175 (+ 1771us) client_negotiation.cc:240] Negotiation successful
0807 18:14:49.428777 (+ 602us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13259,"thread_start_us":13894,"threads_started":1}
I20260807 18:14:49.432351 3462 heartbeater.cc:347] Connected to a master server at 127.18.154.62:37391
I20260807 18:14:49.432691 3462 heartbeater.cc:464] Registering TS with master...
I20260807 18:14:49.433696 3462 heartbeater.cc:511] Master 127.18.154.62:37391 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.18.154.3@KRBTEST.COM; defaulting to no policy
I20260807 18:14:49.435659 2920 ts_manager.cc:194] Registered new tserver with Master: bc24572293df41fdb3875f6a7ef35369 (127.18.154.2:42927)
I20260807 18:14:49.436851 2920 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.154.2@KRBTEST.COM'} at 127.18.154.2:47587
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.154.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.154.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.3.keytab.
Entry for principal kudu/127.18.154.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.3.keytab.
I20260807 18:14:49.481667 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126328808105-19048-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126328808105-19048-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126328808105-19048-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126328808105-19048-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.154.20:59295/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.154.3:0
--local_ip_for_outbound_sockets=127.18.154.3
--webserver_interface=127.18.154.3
--webserver_port=0
--tserver_master_addrs=127.18.154.62:37391
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.3.keytab
--principal=kudu/127.18.154.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.154.20:39619
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126328808105-19048-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126328808105-19048-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126328808105-19048-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 18:14:49.572734 3229 heartbeater.cc:503] Master 127.18.154.62:37391 was elected leader, sending a full tablet report...
W20260807 18:14:49.917719 3469 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:14:49.918085 3469 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:14:49.918255 3469 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:14:49.926887 3469 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:14:49.927208 3469 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:14:49.927465 3469 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.154.3
I20260807 18:14:49.938891 3469 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.154.20:39619
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126328808105-19048-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126328808105-19048-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.18.154.20:59295/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.3.keytab
--principal=kudu/127.18.154.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.154.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126328808105-19048-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.154.3
--webserver_port=0
--tserver_master_addrs=127.18.154.62:37391
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.154.3
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126328808105-19048-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:14:49.940351 3469 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:14:49.941952 3469 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:14:49.957005 3476 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:14:49.957641 3480 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:14:49.957751 3477 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:14:49.959183 3469 server_base.cc:1034] running on GCE node
I20260807 18:14:49.959868 3469 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:14:49.960978 3469 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:14:49.962220 3469 hybrid_clock.cc:648] HybridClock initialized: now 1786126489962172 us; error 38 us; skew 500 ppm
I20260807 18:14:49.962570 3469 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:14:49 dist-test-slave-gkw7 krb5kdc[2234](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126489, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:14:49.966068 3469 init.cc:377] Logged in from keytab as kudu/127.18.154.3@KRBTEST.COM (short username kudu)
I20260807 18:14:49.967697 3469 webserver.cc:460] Webserver started at http://127.18.154.3:41751/ using document root <none> and password file <none>
I20260807 18:14:49.968458 3469 fs_manager.cc:359] Metadata directory not provided
I20260807 18:14:49.968560 3469 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:14:49.968835 3469 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 18:14:49 dist-test-slave-gkw7 krb5kdc[2234](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126489, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.3@KRBTEST.COM for HTTP/127.18.154.20@KRBTEST.COM
I20260807 18:14:49.985144 3469 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126328808105-19048-0/minicluster-data/ts-2/data/instance:
uuid: "4528d9c41dc14edf8387b2711d9020cd"
format_stamp: "Formatted at 2026-08-07 18:14:49 on dist-test-slave-gkw7"
server_key: "7e0392ffe898a94cd6b72bfd33bb6171"
server_key_iv: "db32a803261be0c3bb09a89327602d1b"
server_key_version: "kuduclusterkey@0"
I20260807 18:14:49.986274 3469 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126328808105-19048-0/minicluster-data/ts-2/wal/instance:
uuid: "4528d9c41dc14edf8387b2711d9020cd"
format_stamp: "Formatted at 2026-08-07 18:14:49 on dist-test-slave-gkw7"
server_key: "7e0392ffe898a94cd6b72bfd33bb6171"
server_key_iv: "db32a803261be0c3bb09a89327602d1b"
server_key_version: "kuduclusterkey@0"
I20260807 18:14:49.992599 3469 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260807 18:14:50.011307 3492 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:14:50.013147 3469 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260807 18:14:50.013357 3469 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126328808105-19048-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126328808105-19048-0/minicluster-data/ts-2/wal
uuid: "4528d9c41dc14edf8387b2711d9020cd"
format_stamp: "Formatted at 2026-08-07 18:14:49 on dist-test-slave-gkw7"
server_key: "7e0392ffe898a94cd6b72bfd33bb6171"
server_key_iv: "db32a803261be0c3bb09a89327602d1b"
server_key_version: "kuduclusterkey@0"
I20260807 18:14:50.013652 3469 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126328808105-19048-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126328808105-19048-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126328808105-19048-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:14:50.035849 3469 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:14:50.040411 3469 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:14:50.040802 3469 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:14:50.042075 3469 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:14:50.043489 3469 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:14:50.043568 3469 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:14:50.043658 3469 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:14:50.043711 3469 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:14:50.086652 3469 rpc_server.cc:307] RPC server started. Bound to: 127.18.154.3:39061
I20260807 18:14:50.086841 3650 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.154.3:39061 every 8 connection(s)
I20260807 18:14:50.088236 3469 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126328808105-19048-0/minicluster-data/ts-2/data/info.pb
I20260807 18:14:50.088608 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 3469
I20260807 18:14:50.088796 19048 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126328808105-19048-0/minicluster-data/ts-2/wal/instance
I20260807 18:14:50.089637 19048 external_mini_cluster.cc:1428] Setting key 5429b8d5c2b28366fc9d01d719914b5b
Aug 07 18:14:50 dist-test-slave-gkw7 krb5kdc[2234](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126489, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.3@KRBTEST.COM for kudu/127.18.154.62@KRBTEST.COM
I20260807 18:14:50.112262 3655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:50.091592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.3:51461 (local address 127.18.154.62:37391)
0807 18:14:50.095238 (+ 3646us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:50.095243 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:50.095259 (+ 16us) server_negotiation.cc:407] Connection header received
0807 18:14:50.096872 (+ 1613us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:50.096876 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:50.096944 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:50.097044 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:14:50.098701 (+ 1657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:50.099538 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:50.101323 (+ 1785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:50.102028 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:50.102151 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:50.105980 (+ 3829us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:14:50.105988 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:14:50.105997 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:14:50.106038 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:14:50.108948 (+ 2910us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:50.109769 (+ 821us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:50.109774 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:50.109778 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:50.109913 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:50.110381 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:50.110386 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:50.110389 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:50.110691 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:14:50.110861 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:50.111627 (+ 766us) server_negotiation.cc:299] Negotiation successful
0807 18:14:50.111871 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3552,"thread_start_us":3434,"threads_started":1}
I20260807 18:14:50.113247 3657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:50.092355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:37391 (local address 127.18.154.3:51461)
0807 18:14:50.094783 (+ 2428us) negotiation.cc:107] Waiting for socket to connect
0807 18:14:50.094847 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0807 18:14:50.096591 (+ 1744us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:14:50.097292 (+ 701us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:14:50.097301 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:14:50.097898 (+ 597us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:14:50.098497 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:50.098515 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:50.099678 (+ 1163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:50.099682 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:50.101147 (+ 1465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:50.101168 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:50.102244 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:50.102258 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:50.102476 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:50.103115 (+ 639us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:14:50.103142 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:14:50.105767 (+ 2625us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:14:50.109111 (+ 3344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:14:50.109120 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:14:50.109139 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:14:50.109644 (+ 505us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:14:50.110028 (+ 384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:14:50.110033 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:14:50.110037 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:14:50.110256 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:14:50.110857 (+ 601us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:14:50.110865 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:14:50.111258 (+ 393us) client_negotiation.cc:769] Sending connection context
0807 18:14:50.111538 (+ 280us) client_negotiation.cc:240] Negotiation successful
0807 18:14:50.112089 (+ 551us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2082,"thread_start_us":943,"threads_started":1}
I20260807 18:14:50.114796 3652 heartbeater.cc:347] Connected to a master server at 127.18.154.62:37391
I20260807 18:14:50.115213 3652 heartbeater.cc:464] Registering TS with master...
I20260807 18:14:50.116030 3652 heartbeater.cc:511] Master 127.18.154.62:37391 requested a full tablet report, sending...
I20260807 18:14:50.118170 2919 ts_manager.cc:194] Registered new tserver with Master: 4528d9c41dc14edf8387b2711d9020cd (127.18.154.3:39061)
I20260807 18:14:50.119328 19048 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 18:14:50.119455 2919 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.154.3@KRBTEST.COM'} at 127.18.154.3:51461
I20260807 18:14:50.138496 3655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:50.124974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56380 (local address 127.18.154.62:37391)
0807 18:14:50.125169 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:50.125174 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:50.125868 (+ 694us) server_negotiation.cc:407] Connection header received
0807 18:14:50.126078 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:50.126082 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:50.126168 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:50.126255 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:14:50.127750 (+ 1495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:50.128499 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:50.130100 (+ 1601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:50.130582 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:50.130710 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:50.133245 (+ 2535us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:14:50.133249 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:14:50.133253 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:14:50.133285 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:14:50.136146 (+ 2861us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:50.136761 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:50.136765 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:50.136767 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:50.136894 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:50.137238 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:50.137241 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:50.137243 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:50.137520 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:14:50.137795 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:50.138082 (+ 287us) server_negotiation.cc:299] Negotiation successful
0807 18:14:50.138267 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260807 18:14:50.142922 2919 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56380:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 18:14:50.151121 2919 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:14:50.186028 3672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:50.165292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36918 (local address 127.18.154.1:44579)
0807 18:14:50.175384 (+ 10092us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:50.175398 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:50.176979 (+ 1581us) server_negotiation.cc:407] Connection header received
0807 18:14:50.177107 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:50.177117 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:50.177429 (+ 312us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:50.177754 (+ 325us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:14:50.179208 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:50.181037 (+ 1829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:50.183571 (+ 2534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:50.184656 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:50.184832 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:50.185099 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:50.185354 (+ 255us) server_negotiation.cc:299] Negotiation successful
0807 18:14:50.185597 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9815,"thread_start_us":10098,"threads_started":1}
I20260807 18:14:50.186005 3669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:50.165063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.1:44579 (local address 127.0.0.1:36918)
0807 18:14:50.176445 (+ 11382us) negotiation.cc:107] Waiting for socket to connect
0807 18:14:50.176489 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 18:14:50.176732 (+ 243us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:14:50.177787 (+ 1055us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:14:50.177794 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:14:50.177826 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:14:50.178313 (+ 487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:50.178327 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:50.181188 (+ 2861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:50.181194 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:50.183401 (+ 2207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:50.183414 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:50.184889 (+ 1475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:50.184896 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:50.185092 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:50.185157 (+ 65us) client_negotiation.cc:769] Sending connection context
0807 18:14:50.185307 (+ 150us) client_negotiation.cc:240] Negotiation successful
0807 18:14:50.185493 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11029,"thread_start_us":12568,"threads_started":1}
I20260807 18:14:50.187426 3670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:50.165057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.3:39061 (local address 127.0.0.1:48968)
0807 18:14:50.177914 (+ 12857us) negotiation.cc:107] Waiting for socket to connect
0807 18:14:50.177944 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0807 18:14:50.178075 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:14:50.178813 (+ 738us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:14:50.178817 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:14:50.178857 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:14:50.179409 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:50.179426 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:50.182068 (+ 2642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:50.182074 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:50.184560 (+ 2486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:50.184574 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:50.186568 (+ 1994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:50.186575 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:50.186742 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:50.186786 (+ 44us) client_negotiation.cc:769] Sending connection context
0807 18:14:50.186854 (+ 68us) client_negotiation.cc:240] Negotiation successful
0807 18:14:50.187015 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12718,"mutex_wait_us":27,"thread_start_us":10781,"threads_started":1}
I20260807 18:14:50.188315 3673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:50.165506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48968 (local address 127.18.154.3:39061)
0807 18:14:50.177392 (+ 11886us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:50.177405 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:50.178071 (+ 666us) server_negotiation.cc:407] Connection header received
0807 18:14:50.178229 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:50.178238 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:50.178539 (+ 301us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:50.179622 (+ 1083us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:14:50.179916 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:50.181876 (+ 1960us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:50.184762 (+ 2886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:50.186320 (+ 1558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:50.187439 (+ 1119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:50.187683 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:50.187757 (+ 74us) server_negotiation.cc:299] Negotiation successful
0807 18:14:50.187908 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11637,"thread_start_us":11827,"threads_started":1}
I20260807 18:14:50.188935 3674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:50.178328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51520 (local address 127.18.154.2:42927)
0807 18:14:50.179003 (+ 675us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:50.179012 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:50.179795 (+ 783us) server_negotiation.cc:407] Connection header received
0807 18:14:50.179995 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:50.180000 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:50.180156 (+ 156us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:50.180332 (+ 176us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:14:50.180837 (+ 505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:50.182389 (+ 1552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:50.185010 (+ 2621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:50.186408 (+ 1398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:50.186520 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:50.186741 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:50.188293 (+ 1552us) server_negotiation.cc:299] Negotiation successful
0807 18:14:50.188532 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":522,"thread_start_us":422,"threads_started":1}
I20260807 18:14:50.189277 3675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:50.179050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.2:42927 (local address 127.0.0.1:51520)
0807 18:14:50.179670 (+ 620us) negotiation.cc:107] Waiting for socket to connect
0807 18:14:50.179685 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 18:14:50.179830 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:14:50.180334 (+ 504us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:14:50.180339 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:14:50.180361 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:14:50.180681 (+ 320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:50.180690 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:50.182568 (+ 1878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:50.182586 (+ 18us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:50.184837 (+ 2251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:50.184853 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:50.186988 (+ 2135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:50.186994 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:50.187167 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:50.187199 (+ 32us) client_negotiation.cc:769] Sending connection context
0807 18:14:50.188780 (+ 1581us) client_negotiation.cc:240] Negotiation successful
0807 18:14:50.188903 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":493,"mutex_wait_us":1,"thread_start_us":363,"threads_started":1}
I20260807 18:14:50.190620 3131 tablet_service.cc:1467] Processing CreateTablet for tablet 53bf91e16ded4788a3f2d243e89d362a (DEFAULT_TABLE table=test-table [id=cfabad1647c346b1ac00e2862361c244]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:14:50.192359 3131 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 53bf91e16ded4788a3f2d243e89d362a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:14:50.192466 3353 tablet_service.cc:1467] Processing CreateTablet for tablet 53bf91e16ded4788a3f2d243e89d362a (DEFAULT_TABLE table=test-table [id=cfabad1647c346b1ac00e2862361c244]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:14:50.192421 3562 tablet_service.cc:1467] Processing CreateTablet for tablet 53bf91e16ded4788a3f2d243e89d362a (DEFAULT_TABLE table=test-table [id=cfabad1647c346b1ac00e2862361c244]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:14:50.193871 3353 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 53bf91e16ded4788a3f2d243e89d362a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:14:50.193946 3562 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 53bf91e16ded4788a3f2d243e89d362a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:14:50.206353 3678 tablet_bootstrap.cc:492] T 53bf91e16ded4788a3f2d243e89d362a P bc24572293df41fdb3875f6a7ef35369: Bootstrap starting.
I20260807 18:14:50.209568 3682 tablet_bootstrap.cc:492] T 53bf91e16ded4788a3f2d243e89d362a P 4528d9c41dc14edf8387b2711d9020cd: Bootstrap starting.
I20260807 18:14:50.210098 3678 tablet_bootstrap.cc:654] T 53bf91e16ded4788a3f2d243e89d362a P bc24572293df41fdb3875f6a7ef35369: Neither blocks nor log segments found. Creating new log.
I20260807 18:14:50.211016 3681 tablet_bootstrap.cc:492] T 53bf91e16ded4788a3f2d243e89d362a P d9cb0ea1ee744b7bbc3e975c7c817b22: Bootstrap starting.
I20260807 18:14:50.211881 3678 log.cc:824] T 53bf91e16ded4788a3f2d243e89d362a P bc24572293df41fdb3875f6a7ef35369: Log is configured to *not* fsync() on all Append() calls
I20260807 18:14:50.214123 3682 tablet_bootstrap.cc:654] T 53bf91e16ded4788a3f2d243e89d362a P 4528d9c41dc14edf8387b2711d9020cd: Neither blocks nor log segments found. Creating new log.
I20260807 18:14:50.214845 3681 tablet_bootstrap.cc:654] T 53bf91e16ded4788a3f2d243e89d362a P d9cb0ea1ee744b7bbc3e975c7c817b22: Neither blocks nor log segments found. Creating new log.
I20260807 18:14:50.215514 3682 log.cc:824] T 53bf91e16ded4788a3f2d243e89d362a P 4528d9c41dc14edf8387b2711d9020cd: Log is configured to *not* fsync() on all Append() calls
I20260807 18:14:50.215984 3681 log.cc:824] T 53bf91e16ded4788a3f2d243e89d362a P d9cb0ea1ee744b7bbc3e975c7c817b22: Log is configured to *not* fsync() on all Append() calls
I20260807 18:14:50.216904 3678 tablet_bootstrap.cc:492] T 53bf91e16ded4788a3f2d243e89d362a P bc24572293df41fdb3875f6a7ef35369: No bootstrap required, opened a new log
I20260807 18:14:50.217250 3678 ts_tablet_manager.cc:1397] T 53bf91e16ded4788a3f2d243e89d362a P bc24572293df41fdb3875f6a7ef35369: Time spent bootstrapping tablet: real 0.011s user 0.000s sys 0.009s
I20260807 18:14:50.219170 3681 tablet_bootstrap.cc:492] T 53bf91e16ded4788a3f2d243e89d362a P d9cb0ea1ee744b7bbc3e975c7c817b22: No bootstrap required, opened a new log
I20260807 18:14:50.219529 3681 ts_tablet_manager.cc:1397] T 53bf91e16ded4788a3f2d243e89d362a P d9cb0ea1ee744b7bbc3e975c7c817b22: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.005s
I20260807 18:14:50.220291 3682 tablet_bootstrap.cc:492] T 53bf91e16ded4788a3f2d243e89d362a P 4528d9c41dc14edf8387b2711d9020cd: No bootstrap required, opened a new log
I20260807 18:14:50.220706 3682 ts_tablet_manager.cc:1397] T 53bf91e16ded4788a3f2d243e89d362a P 4528d9c41dc14edf8387b2711d9020cd: Time spent bootstrapping tablet: real 0.011s user 0.010s sys 0.000s
I20260807 18:14:50.227519 3681 raft_consensus.cc:348] T 53bf91e16ded4788a3f2d243e89d362a P d9cb0ea1ee744b7bbc3e975c7c817b22 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9cb0ea1ee744b7bbc3e975c7c817b22" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 44579 } } peers { permanent_uuid: "4528d9c41dc14edf8387b2711d9020cd" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 39061 } } peers { permanent_uuid: "bc24572293df41fdb3875f6a7ef35369" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 42927 } }
I20260807 18:14:50.227952 3681 raft_consensus.cc:374] T 53bf91e16ded4788a3f2d243e89d362a P d9cb0ea1ee744b7bbc3e975c7c817b22 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:14:50.228109 3681 raft_consensus.cc:729] T 53bf91e16ded4788a3f2d243e89d362a P d9cb0ea1ee744b7bbc3e975c7c817b22 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9cb0ea1ee744b7bbc3e975c7c817b22, State: Initialized, Role: FOLLOWER
I20260807 18:14:50.228843 3681 consensus_queue.cc:260] T 53bf91e16ded4788a3f2d243e89d362a P d9cb0ea1ee744b7bbc3e975c7c817b22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9cb0ea1ee744b7bbc3e975c7c817b22" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 44579 } } peers { permanent_uuid: "4528d9c41dc14edf8387b2711d9020cd" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 39061 } } peers { permanent_uuid: "bc24572293df41fdb3875f6a7ef35369" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 42927 } }
I20260807 18:14:50.229087 3678 raft_consensus.cc:348] T 53bf91e16ded4788a3f2d243e89d362a P bc24572293df41fdb3875f6a7ef35369 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9cb0ea1ee744b7bbc3e975c7c817b22" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 44579 } } peers { permanent_uuid: "4528d9c41dc14edf8387b2711d9020cd" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 39061 } } peers { permanent_uuid: "bc24572293df41fdb3875f6a7ef35369" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 42927 } }
I20260807 18:14:50.229907 3678 raft_consensus.cc:374] T 53bf91e16ded4788a3f2d243e89d362a P bc24572293df41fdb3875f6a7ef35369 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:14:50.230063 3678 raft_consensus.cc:729] T 53bf91e16ded4788a3f2d243e89d362a P bc24572293df41fdb3875f6a7ef35369 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc24572293df41fdb3875f6a7ef35369, State: Initialized, Role: FOLLOWER
I20260807 18:14:50.230650 3681 ts_tablet_manager.cc:1428] T 53bf91e16ded4788a3f2d243e89d362a P d9cb0ea1ee744b7bbc3e975c7c817b22: Time spent starting tablet: real 0.011s user 0.006s sys 0.004s
I20260807 18:14:50.230865 3678 consensus_queue.cc:260] T 53bf91e16ded4788a3f2d243e89d362a P bc24572293df41fdb3875f6a7ef35369 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9cb0ea1ee744b7bbc3e975c7c817b22" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 44579 } } peers { permanent_uuid: "4528d9c41dc14edf8387b2711d9020cd" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 39061 } } peers { permanent_uuid: "bc24572293df41fdb3875f6a7ef35369" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 42927 } }
I20260807 18:14:50.231997 3462 heartbeater.cc:503] Master 127.18.154.62:37391 was elected leader, sending a full tablet report...
I20260807 18:14:50.232008 3682 raft_consensus.cc:348] T 53bf91e16ded4788a3f2d243e89d362a P 4528d9c41dc14edf8387b2711d9020cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9cb0ea1ee744b7bbc3e975c7c817b22" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 44579 } } peers { permanent_uuid: "4528d9c41dc14edf8387b2711d9020cd" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 39061 } } peers { permanent_uuid: "bc24572293df41fdb3875f6a7ef35369" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 42927 } }
I20260807 18:14:50.232441 3682 raft_consensus.cc:374] T 53bf91e16ded4788a3f2d243e89d362a P 4528d9c41dc14edf8387b2711d9020cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:14:50.232548 3682 raft_consensus.cc:729] T 53bf91e16ded4788a3f2d243e89d362a P 4528d9c41dc14edf8387b2711d9020cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4528d9c41dc14edf8387b2711d9020cd, State: Initialized, Role: FOLLOWER
I20260807 18:14:50.233166 3682 consensus_queue.cc:260] T 53bf91e16ded4788a3f2d243e89d362a P 4528d9c41dc14edf8387b2711d9020cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9cb0ea1ee744b7bbc3e975c7c817b22" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 44579 } } peers { permanent_uuid: "4528d9c41dc14edf8387b2711d9020cd" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 39061 } } peers { permanent_uuid: "bc24572293df41fdb3875f6a7ef35369" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 42927 } }
I20260807 18:14:50.234459 3652 heartbeater.cc:503] Master 127.18.154.62:37391 was elected leader, sending a full tablet report...
I20260807 18:14:50.235049 3682 ts_tablet_manager.cc:1428] T 53bf91e16ded4788a3f2d243e89d362a P 4528d9c41dc14edf8387b2711d9020cd: Time spent starting tablet: real 0.014s user 0.007s sys 0.007s
I20260807 18:14:50.236774 3678 ts_tablet_manager.cc:1428] T 53bf91e16ded4788a3f2d243e89d362a P bc24572293df41fdb3875f6a7ef35369: Time spent starting tablet: real 0.019s user 0.009s sys 0.005s
W20260807 18:14:50.266333 3230 tablet.cc:2369] T 53bf91e16ded4788a3f2d243e89d362a P d9cb0ea1ee744b7bbc3e975c7c817b22: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 18:14:50.340633 3654 tablet.cc:2369] T 53bf91e16ded4788a3f2d243e89d362a P 4528d9c41dc14edf8387b2711d9020cd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:14:50.341424 3687 raft_consensus.cc:482] T 53bf91e16ded4788a3f2d243e89d362a P d9cb0ea1ee744b7bbc3e975c7c817b22 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:14:50.341826 3687 raft_consensus.cc:504] T 53bf91e16ded4788a3f2d243e89d362a P d9cb0ea1ee744b7bbc3e975c7c817b22 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9cb0ea1ee744b7bbc3e975c7c817b22" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 44579 } } peers { permanent_uuid: "4528d9c41dc14edf8387b2711d9020cd" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 39061 } } peers { permanent_uuid: "bc24572293df41fdb3875f6a7ef35369" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 42927 } }
I20260807 18:14:50.343667 3687 leader_election.cc:288] T 53bf91e16ded4788a3f2d243e89d362a P d9cb0ea1ee744b7bbc3e975c7c817b22 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4528d9c41dc14edf8387b2711d9020cd (127.18.154.3:39061), bc24572293df41fdb3875f6a7ef35369 (127.18.154.2:42927)
I20260807 18:14:50.353294 3674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:50.344405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.1:53417 (local address 127.18.154.2:42927)
0807 18:14:50.344629 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:50.344637 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:50.345423 (+ 786us) server_negotiation.cc:407] Connection header received
0807 18:14:50.345613 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:50.345618 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:50.345722 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:50.345792 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:14:50.346543 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:50.347656 (+ 1113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:50.350223 (+ 2567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:50.351870 (+ 1647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:50.352033 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:50.352113 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:50.352867 (+ 754us) server_negotiation.cc:299] Negotiation successful
0807 18:14:50.353002 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 18:14:50.353286 3673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:50.343951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.1:42999 (local address 127.18.154.3:39061)
0807 18:14:50.344223 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:50.344233 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:50.345053 (+ 820us) server_negotiation.cc:407] Connection header received
0807 18:14:50.345315 (+ 262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:50.345320 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:50.345404 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:50.345504 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:14:50.346189 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:50.347374 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:50.350223 (+ 2849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:50.351854 (+ 1631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:50.352064 (+ 210us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:50.352130 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:50.352867 (+ 737us) server_negotiation.cc:299] Negotiation successful
0807 18:14:50.353001 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260807 18:14:50.353628 3690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:50.343907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.3:39061 (local address 127.18.154.1:42999)
0807 18:14:50.344927 (+ 1020us) negotiation.cc:107] Waiting for socket to connect
0807 18:14:50.344948 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 18:14:50.345153 (+ 205us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:14:50.345574 (+ 421us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:14:50.345579 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:14:50.345607 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:14:50.346015 (+ 408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:50.346033 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:50.347515 (+ 1482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:50.347519 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:50.349980 (+ 2461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:50.350002 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:50.352368 (+ 2366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:50.352374 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:50.352709 (+ 335us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:50.352735 (+ 26us) client_negotiation.cc:769] Sending connection context
0807 18:14:50.352814 (+ 79us) client_negotiation.cc:240] Negotiation successful
0807 18:14:50.352997 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":885,"mutex_wait_us":79,"thread_start_us":727,"threads_started":1}
I20260807 18:14:50.353652 3691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:50.344160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.2:42927 (local address 127.18.154.1:53417)
0807 18:14:50.345306 (+ 1146us) negotiation.cc:107] Waiting for socket to connect
0807 18:14:50.345321 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 18:14:50.345441 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:14:50.345840 (+ 399us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:14:50.345846 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:14:50.345879 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:14:50.346379 (+ 500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:50.346390 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:50.347771 (+ 1381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:50.347778 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:50.350035 (+ 2257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:14:50.350047 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:14:50.352248 (+ 2201us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:14:50.352270 (+ 22us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:14:50.352526 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:50.352702 (+ 176us) client_negotiation.cc:769] Sending connection context
0807 18:14:50.352810 (+ 108us) client_negotiation.cc:240] Negotiation successful
0807 18:14:50.352997 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1026,"mutex_wait_us":86,"thread_start_us":736,"threads_started":1}
I20260807 18:14:50.355101 3382 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "53bf91e16ded4788a3f2d243e89d362a" candidate_uuid: "d9cb0ea1ee744b7bbc3e975c7c817b22" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc24572293df41fdb3875f6a7ef35369" is_pre_election: true
I20260807 18:14:50.355126 3588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "53bf91e16ded4788a3f2d243e89d362a" candidate_uuid: "d9cb0ea1ee744b7bbc3e975c7c817b22" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4528d9c41dc14edf8387b2711d9020cd" is_pre_election: true
I20260807 18:14:50.355609 3382 raft_consensus.cc:2454] T 53bf91e16ded4788a3f2d243e89d362a P bc24572293df41fdb3875f6a7ef35369 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cb0ea1ee744b7bbc3e975c7c817b22 in term 0.
I20260807 18:14:50.355648 3588 raft_consensus.cc:2454] T 53bf91e16ded4788a3f2d243e89d362a P 4528d9c41dc14edf8387b2711d9020cd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cb0ea1ee744b7bbc3e975c7c817b22 in term 0.
I20260807 18:14:50.356335 3062 leader_election.cc:302] T 53bf91e16ded4788a3f2d243e89d362a P d9cb0ea1ee744b7bbc3e975c7c817b22 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc24572293df41fdb3875f6a7ef35369, d9cb0ea1ee744b7bbc3e975c7c817b22; no voters:
I20260807 18:14:50.356784 3687 raft_consensus.cc:2781] T 53bf91e16ded4788a3f2d243e89d362a P d9cb0ea1ee744b7bbc3e975c7c817b22 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:14:50.356956 3687 raft_consensus.cc:482] T 53bf91e16ded4788a3f2d243e89d362a P d9cb0ea1ee744b7bbc3e975c7c817b22 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:14:50.357106 3687 raft_consensus.cc:3037] T 53bf91e16ded4788a3f2d243e89d362a P d9cb0ea1ee744b7bbc3e975c7c817b22 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:14:50.359654 3687 raft_consensus.cc:504] T 53bf91e16ded4788a3f2d243e89d362a P d9cb0ea1ee744b7bbc3e975c7c817b22 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9cb0ea1ee744b7bbc3e975c7c817b22" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 44579 } } peers { permanent_uuid: "4528d9c41dc14edf8387b2711d9020cd" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 39061 } } peers { permanent_uuid: "bc24572293df41fdb3875f6a7ef35369" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 42927 } }
I20260807 18:14:50.360383 3687 leader_election.cc:288] T 53bf91e16ded4788a3f2d243e89d362a P d9cb0ea1ee744b7bbc3e975c7c817b22 [CANDIDATE]: Term 1 election: Requested vote from peers 4528d9c41dc14edf8387b2711d9020cd (127.18.154.3:39061), bc24572293df41fdb3875f6a7ef35369 (127.18.154.2:42927)
I20260807 18:14:50.360891 3382 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "53bf91e16ded4788a3f2d243e89d362a" candidate_uuid: "d9cb0ea1ee744b7bbc3e975c7c817b22" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc24572293df41fdb3875f6a7ef35369"
I20260807 18:14:50.360921 3588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "53bf91e16ded4788a3f2d243e89d362a" candidate_uuid: "d9cb0ea1ee744b7bbc3e975c7c817b22" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4528d9c41dc14edf8387b2711d9020cd"
I20260807 18:14:50.361109 3588 raft_consensus.cc:3037] T 53bf91e16ded4788a3f2d243e89d362a P 4528d9c41dc14edf8387b2711d9020cd [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:14:50.361199 3382 raft_consensus.cc:3037] T 53bf91e16ded4788a3f2d243e89d362a P bc24572293df41fdb3875f6a7ef35369 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:14:50.364593 3588 raft_consensus.cc:2454] T 53bf91e16ded4788a3f2d243e89d362a P 4528d9c41dc14edf8387b2711d9020cd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cb0ea1ee744b7bbc3e975c7c817b22 in term 1.
I20260807 18:14:50.364677 3382 raft_consensus.cc:2454] T 53bf91e16ded4788a3f2d243e89d362a P bc24572293df41fdb3875f6a7ef35369 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cb0ea1ee744b7bbc3e975c7c817b22 in term 1.
I20260807 18:14:50.365062 3061 leader_election.cc:302] T 53bf91e16ded4788a3f2d243e89d362a P d9cb0ea1ee744b7bbc3e975c7c817b22 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4528d9c41dc14edf8387b2711d9020cd, d9cb0ea1ee744b7bbc3e975c7c817b22; no voters:
I20260807 18:14:50.365384 3687 raft_consensus.cc:2781] T 53bf91e16ded4788a3f2d243e89d362a P d9cb0ea1ee744b7bbc3e975c7c817b22 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:14:50.365899 3687 raft_consensus.cc:686] T 53bf91e16ded4788a3f2d243e89d362a P d9cb0ea1ee744b7bbc3e975c7c817b22 [term 1 LEADER]: Becoming Leader. State: Replica: d9cb0ea1ee744b7bbc3e975c7c817b22, State: Running, Role: LEADER
I20260807 18:14:50.366427 3687 consensus_queue.cc:237] T 53bf91e16ded4788a3f2d243e89d362a P d9cb0ea1ee744b7bbc3e975c7c817b22 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9cb0ea1ee744b7bbc3e975c7c817b22" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 44579 } } peers { permanent_uuid: "4528d9c41dc14edf8387b2711d9020cd" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 39061 } } peers { permanent_uuid: "bc24572293df41fdb3875f6a7ef35369" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 42927 } }
I20260807 18:14:50.371611 2919 catalog_manager.cc:5495] T 53bf91e16ded4788a3f2d243e89d362a P d9cb0ea1ee744b7bbc3e975c7c817b22 reported cstate change: term changed from 0 to 1, leader changed from <none> to d9cb0ea1ee744b7bbc3e975c7c817b22 (127.18.154.1). New cstate: current_term: 1 leader_uuid: "d9cb0ea1ee744b7bbc3e975c7c817b22" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9cb0ea1ee744b7bbc3e975c7c817b22" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 44579 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4528d9c41dc14edf8387b2711d9020cd" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 39061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc24572293df41fdb3875f6a7ef35369" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 42927 } health_report { overall_health: UNKNOWN } } }
W20260807 18:14:50.399092 3463 tablet.cc:2369] T 53bf91e16ded4788a3f2d243e89d362a P bc24572293df41fdb3875f6a7ef35369: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:14:50.415861 3672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:50.409894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36924 (local address 127.18.154.1:44579)
0807 18:14:50.410131 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:50.410136 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:50.410209 (+ 73us) server_negotiation.cc:407] Connection header received
0807 18:14:50.410389 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:50.410392 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:50.410493 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:50.410621 (+ 128us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:14:50.410965 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:50.411774 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:50.413298 (+ 1524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:50.413779 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:50.413877 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:50.414576 (+ 699us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:14:50.415207 (+ 631us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:14:50.415320 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:50.415440 (+ 120us) server_negotiation.cc:299] Negotiation successful
0807 18:14:50.415507 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260807 18:14:50.424528 3588 raft_consensus.cc:1261] T 53bf91e16ded4788a3f2d243e89d362a P 4528d9c41dc14edf8387b2711d9020cd [term 1 FOLLOWER]: Refusing update from remote peer d9cb0ea1ee744b7bbc3e975c7c817b22: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:14:50.424753 3382 raft_consensus.cc:1261] T 53bf91e16ded4788a3f2d243e89d362a P bc24572293df41fdb3875f6a7ef35369 [term 1 FOLLOWER]: Refusing update from remote peer d9cb0ea1ee744b7bbc3e975c7c817b22: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:14:50.425696 3687 consensus_queue.cc:1035] T 53bf91e16ded4788a3f2d243e89d362a P d9cb0ea1ee744b7bbc3e975c7c817b22 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4528d9c41dc14edf8387b2711d9020cd" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 39061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:14:50.426044 3692 consensus_queue.cc:1035] T 53bf91e16ded4788a3f2d243e89d362a P d9cb0ea1ee744b7bbc3e975c7c817b22 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc24572293df41fdb3875f6a7ef35369" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 42927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:14:50.436540 3699 mvcc.cc:205] Tried to move back new op lower bound from 7315974104768835584 to 7315974104544768000. Current Snapshot: MvccSnapshot[applied={T|T < 7315974104768835584}]
I20260807 18:14:50.437208 3700 mvcc.cc:205] Tried to move back new op lower bound from 7315974104768835584 to 7315974104544768000. Current Snapshot: MvccSnapshot[applied={T|T < 7315974104768835584}]
I20260807 18:14:50.446745 3702 mvcc.cc:205] Tried to move back new op lower bound from 7315974104768835584 to 7315974104544768000. Current Snapshot: MvccSnapshot[applied={T|T < 7315974104768835584}]
I20260807 18:14:50.454174 2919 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56380:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:14:50.454432 2919 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56380:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:14:50.458459 2919 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c6ff551ba7dc4b1995b065b1459f7b02: Sending DeleteTablet for 3 replicas of tablet 53bf91e16ded4788a3f2d243e89d362a
I20260807 18:14:50.459745 3562 tablet_service.cc:1514] Processing DeleteTablet for tablet 53bf91e16ded4788a3f2d243e89d362a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:14:50 UTC) from {username='kudu', principal='kudu/127.18.154.62@KRBTEST.COM'} at 127.0.0.1:48968
I20260807 18:14:50.459759 3353 tablet_service.cc:1514] Processing DeleteTablet for tablet 53bf91e16ded4788a3f2d243e89d362a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:14:50 UTC) from {username='kudu', principal='kudu/127.18.154.62@KRBTEST.COM'} at 127.0.0.1:51520
I20260807 18:14:50.460716 3131 tablet_service.cc:1514] Processing DeleteTablet for tablet 53bf91e16ded4788a3f2d243e89d362a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:14:50 UTC) from {username='kudu', principal='kudu/127.18.154.62@KRBTEST.COM'} at 127.0.0.1:36918
I20260807 18:14:50.461381 3710 tablet_replica.cc:330] T 53bf91e16ded4788a3f2d243e89d362a P bc24572293df41fdb3875f6a7ef35369: stopping tablet replica
I20260807 18:14:50.461603 3713 tablet_replica.cc:330] T 53bf91e16ded4788a3f2d243e89d362a P 4528d9c41dc14edf8387b2711d9020cd: stopping tablet replica
I20260807 18:14:50.461936 3710 raft_consensus.cc:2229] T 53bf91e16ded4788a3f2d243e89d362a P bc24572293df41fdb3875f6a7ef35369 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:14:50.462001 3713 raft_consensus.cc:2229] T 53bf91e16ded4788a3f2d243e89d362a P 4528d9c41dc14edf8387b2711d9020cd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:14:50.462219 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 3034
I20260807 18:14:50.462539 3710 raft_consensus.cc:2258] T 53bf91e16ded4788a3f2d243e89d362a P bc24572293df41fdb3875f6a7ef35369 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:14:50.462517 3713 raft_consensus.cc:2258] T 53bf91e16ded4788a3f2d243e89d362a P 4528d9c41dc14edf8387b2711d9020cd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:14:50.465647 3713 ts_tablet_manager.cc:1905] T 53bf91e16ded4788a3f2d243e89d362a P 4528d9c41dc14edf8387b2711d9020cd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:14:50.467031 3710 ts_tablet_manager.cc:1905] T 53bf91e16ded4788a3f2d243e89d362a P bc24572293df41fdb3875f6a7ef35369: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:14:50.474386 3710 ts_tablet_manager.cc:1918] T 53bf91e16ded4788a3f2d243e89d362a P bc24572293df41fdb3875f6a7ef35369: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:14:50.474634 3710 log.cc:1196] T 53bf91e16ded4788a3f2d243e89d362a P bc24572293df41fdb3875f6a7ef35369: Deleting WAL directory at /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126328808105-19048-0/minicluster-data/ts-1/wal/wals/53bf91e16ded4788a3f2d243e89d362a
I20260807 18:14:50.475291 3710 ts_tablet_manager.cc:1939] T 53bf91e16ded4788a3f2d243e89d362a P bc24572293df41fdb3875f6a7ef35369: Deleting consensus metadata
I20260807 18:14:50.475374 3714 tablet_replica.cc:330] T 53bf91e16ded4788a3f2d243e89d362a P d9cb0ea1ee744b7bbc3e975c7c817b22: stopping tablet replica
I20260807 18:14:50.476006 3714 raft_consensus.cc:2229] T 53bf91e16ded4788a3f2d243e89d362a P d9cb0ea1ee744b7bbc3e975c7c817b22 [term 1 LEADER]: Raft consensus shutting down.
I20260807 18:14:50.476979 3714 raft_consensus.cc:2258] T 53bf91e16ded4788a3f2d243e89d362a P d9cb0ea1ee744b7bbc3e975c7c817b22 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:14:50.477449 2904 catalog_manager.cc:4841] TS bc24572293df41fdb3875f6a7ef35369 (127.18.154.2:42927): tablet 53bf91e16ded4788a3f2d243e89d362a (table test-table [id=cfabad1647c346b1ac00e2862361c244]) successfully deleted
I20260807 18:14:50.479538 3714 ts_tablet_manager.cc:1905] T 53bf91e16ded4788a3f2d243e89d362a P d9cb0ea1ee744b7bbc3e975c7c817b22: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:14:50.479730 3713 ts_tablet_manager.cc:1918] T 53bf91e16ded4788a3f2d243e89d362a P 4528d9c41dc14edf8387b2711d9020cd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:14:50.479930 3713 log.cc:1196] T 53bf91e16ded4788a3f2d243e89d362a P 4528d9c41dc14edf8387b2711d9020cd: Deleting WAL directory at /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126328808105-19048-0/minicluster-data/ts-2/wal/wals/53bf91e16ded4788a3f2d243e89d362a
Aug 07 18:14:50 dist-test-slave-gkw7 krb5kdc[2234](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126485, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.154.1@KRBTEST.COM
I20260807 18:14:50.480515 3713 ts_tablet_manager.cc:1939] T 53bf91e16ded4788a3f2d243e89d362a P 4528d9c41dc14edf8387b2711d9020cd: Deleting consensus metadata
I20260807 18:14:50.482695 2903 catalog_manager.cc:4841] TS 4528d9c41dc14edf8387b2711d9020cd (127.18.154.3:39061): tablet 53bf91e16ded4788a3f2d243e89d362a (table test-table [id=cfabad1647c346b1ac00e2862361c244]) successfully deleted
I20260807 18:14:50.488036 3714 ts_tablet_manager.cc:1918] T 53bf91e16ded4788a3f2d243e89d362a P d9cb0ea1ee744b7bbc3e975c7c817b22: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:14:50.488281 3714 log.cc:1196] T 53bf91e16ded4788a3f2d243e89d362a P d9cb0ea1ee744b7bbc3e975c7c817b22: Deleting WAL directory at /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786126328808105-19048-0/minicluster-data/ts-0/wal/wals/53bf91e16ded4788a3f2d243e89d362a
I20260807 18:14:50.488904 3714 ts_tablet_manager.cc:1939] T 53bf91e16ded4788a3f2d243e89d362a P d9cb0ea1ee744b7bbc3e975c7c817b22: Deleting consensus metadata
I20260807 18:14:50.489806 3672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:50.463053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36936 (local address 127.18.154.1:44579)
0807 18:14:50.463268 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:50.463278 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:50.464138 (+ 860us) server_negotiation.cc:407] Connection header received
0807 18:14:50.464374 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:50.464378 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:50.464438 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:50.464542 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:14:50.466171 (+ 1629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:50.467399 (+ 1228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:50.469368 (+ 1969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:50.470132 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:50.470240 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:50.481512 (+ 11272us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:14:50.481569 (+ 57us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:14:50.481588 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:14:50.481648 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:14:50.486118 (+ 4470us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:50.486943 (+ 825us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:50.486950 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:50.486953 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:50.487113 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:50.487523 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:50.487530 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:50.487534 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:50.488019 (+ 485us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:14:50.488417 (+ 398us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:50.488955 (+ 538us) server_negotiation.cc:299] Negotiation successful
0807 18:14:50.489337 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260807 18:14:50.490638 3220 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:14:50.490960 2904 catalog_manager.cc:4841] TS d9cb0ea1ee744b7bbc3e975c7c817b22 (127.18.154.1:44579): tablet 53bf91e16ded4788a3f2d243e89d362a (table test-table [id=cfabad1647c346b1ac00e2862361c244]) successfully deleted
I20260807 18:14:50.590472 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 3034
I20260807 18:14:50.616722 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 3238
Aug 07 18:14:50 dist-test-slave-gkw7 krb5kdc[2234](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126485, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.154.2@KRBTEST.COM
I20260807 18:14:50.633021 3674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:50.617678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51528 (local address 127.18.154.2:42927)
0807 18:14:50.617884 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:50.617892 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:50.617922 (+ 30us) server_negotiation.cc:407] Connection header received
0807 18:14:50.618364 (+ 442us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:50.618371 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:50.618449 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:50.618564 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:14:50.619949 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:50.620816 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:50.622278 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:50.622680 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:50.622820 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:50.627326 (+ 4506us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:14:50.627342 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:14:50.627359 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:14:50.627402 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:14:50.630258 (+ 2856us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:50.631010 (+ 752us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:50.631018 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:50.631020 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:50.631189 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:50.631551 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:50.631555 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:50.631557 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:50.631972 (+ 415us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:14:50.632259 (+ 287us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:50.632547 (+ 288us) server_negotiation.cc:299] Negotiation successful
0807 18:14:50.632727 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260807 18:14:50.633900 3454 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:14:50.726068 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 3238
I20260807 18:14:50.752197 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 3469
Aug 07 18:14:50 dist-test-slave-gkw7 krb5kdc[2234](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126485, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.154.3@KRBTEST.COM
I20260807 18:14:50.768823 3673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:14:50.753052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48970 (local address 127.18.154.3:39061)
0807 18:14:50.753323 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0807 18:14:50.753331 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:14:50.753354 (+ 23us) server_negotiation.cc:407] Connection header received
0807 18:14:50.753603 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:14:50.753610 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:14:50.753698 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:14:50.753841 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:14:50.755331 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:50.756104 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:50.757586 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:14:50.758070 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:14:50.758228 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:14:50.762963 (+ 4735us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:14:50.762980 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:14:50.762998 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:14:50.763041 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:14:50.765963 (+ 2922us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:50.766678 (+ 715us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:50.766691 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:50.766695 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:50.766836 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:14:50.767232 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:14:50.767238 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:14:50.767241 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:14:50.767604 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:14:50.767908 (+ 304us) server_negotiation.cc:1035] Waiting for connection context
0807 18:14:50.768261 (+ 353us) server_negotiation.cc:299] Negotiation successful
0807 18:14:50.768488 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121}
I20260807 18:14:50.769522 3642 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:14:50.858367 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 3469
I20260807 18:14:50.885353 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 2875
I20260807 18:14:50.886790 2962 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:14:51.001789 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 2875
I20260807 18:14:51.045672 19048 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260807 18:14:51.403913 19048 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260807 18:14:51.404248 19048 mini_ranger.cc:66] Stopping Ranger...
I20260807 18:14:51.750379 19048 mini_ranger.cc:68] Stopped Ranger
2026-08-07 18:14:51.750 UTC [2280] LOG: received smart shutdown request
2026-08-07 18:14:51.755 UTC [2280] LOG: background worker "logical replication launcher" (PID 2291) exited with exit code 1
2026-08-07 18:14:51.755 UTC [2284] LOG: shutting down
2026-08-07 18:14:51.936 UTC [2280] LOG: database system is shut down
2026-08-07T18:14:51Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (53348 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:14:52 dist-test-slave-gkw7 krb5kdc[3738](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:14:52 dist-test-slave-gkw7 krb5kdc[3738](info): set up 2 sockets
Aug 07 18:14:52 dist-test-slave-gkw7 krb5kdc[3738](info): commencing operation
krb5kdc: starting...
W20260807 18:14:53.405795 19048 mini_kdc.cc:121] Time spent starting KDC: real 1.288s user 0.010s sys 0.028s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/krb5kdc/test-user.keytab.
Aug 07 18:14:53 dist-test-slave-gkw7 krb5kdc[3738](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126493, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:14:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:14:53Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.18.154.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.18.154.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.18.154.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/krb5kdc/rangeradmin_127.18.154.20@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.18.154.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/krb5kdc/rangeradmin_127.18.154.20@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.18.154.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.18.154.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.18.154.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/krb5kdc/rangerlookup_127.18.154.20@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.18.154.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/krb5kdc/rangerlookup_127.18.154.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.18.154.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.18.154.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.18.154.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/krb5kdc/HTTP_127.18.154.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.18.154.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/krb5kdc/HTTP_127.18.154.20@KRBTEST.COM.keytab.
I20260807 18:14:53.829447 19048 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskTVw2hL/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/minicluster-data/postgres -l logfile start
2026-08-07 18:14:56.552 UTC [3776] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-07 18:14:56.552 UTC [3776] LOG: listening on IPv4 address "127.18.154.20", port 42415
2026-08-07 18:14:56.556 UTC [3776] LOG: listening on Unix socket "/tmp/.s.PGSQL.42415"
2026-08-07 18:14:56.582 UTC [3779] LOG: database system was shut down at 2026-08-07 18:14:54 UTC
2026-08-07 18:14:56.586 UTC [3776] LOG: database system is ready to accept connections
I20260807 18:14:57.754866 19048 mini_postgres.cc:96] Postgres bound to 42415
2026-08-07 18:14:57.779 UTC [3792] FATAL: database "slave" does not exist
127.18.154.20:42415 - accepting connections
I20260807 18:14:57.780562 19048 mini_ranger.cc:161] Starting Ranger...
I20260807 18:14:57.813647 19048 mini_ranger.cc:84] Created miniranger Postgres user
I20260807 18:14:58.358879 19048 mini_ranger.cc:87] Created ranger Postgres database
I20260807 18:14:58.359026 19048 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/minicluster-data/ranger-admin
2026-08-07 18:14:58,682 [I] DB FLAVOR :POSTGRES
2026-08-07 18:14:58,682 [I] --------- Verifying Ranger DB connection ---------
2026-08-07 18:14:58,682 [I] Checking connection..
2026-08-07 18:14:58,682 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTVw2hL/build/asan/bin/postgresql.jar:/tmp/dist-test-taskTVw2hL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.154.20:42415/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-07 18:14:58,984 [I] Checking connection passed.
2026-08-07 18:14:58,984 [I] --------- Verifying version history table ---------
2026-08-07 18:14:58,984 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTVw2hL/build/asan/bin/postgresql.jar:/tmp/dist-test-taskTVw2hL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.154.20:42415/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 18:14:59,248 [I] Table x_db_version_h does not exist in database ranger
2026-08-07 18:14:59,248 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTVw2hL/build/asan/bin/postgresql.jar:/tmp/dist-test-taskTVw2hL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.154.20:42415/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 18:14:59,517 [I] Table x_db_version_h does not exist in database ranger
2026-08-07 18:14:59,517 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-07 18:14:59,518 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTVw2hL/build/asan/bin/postgresql.jar:/tmp/dist-test-taskTVw2hL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.154.20:42415/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskTVw2hL/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-07 18:14:59.758 UTC [3884] WARNING: there is no transaction in progress
2026-08-07 18:14:59,766 [I] create_dbversion_catalog.sql file imported successfully
2026-08-07 18:14:59,766 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTVw2hL/build/asan/bin/postgresql.jar:/tmp/dist-test-taskTVw2hL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.154.20:42415/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 18:15:00,031 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-07 18:15:00,031 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-07 18:15:00,032 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTVw2hL/build/asan/bin/postgresql.jar:/tmp/dist-test-taskTVw2hL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.154.20:42415/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-07 18:15:00,318 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTVw2hL/build/asan/bin/postgresql.jar:/tmp/dist-test-taskTVw2hL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.154.20:42415/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-07 18:15:00,581 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTVw2hL/build/asan/bin/postgresql.jar:/tmp/dist-test-taskTVw2hL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.154.20:42415/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-gkw7.c.gcp-upstream.internal','N') ;"
2026-08-07 18:15:00,845 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTVw2hL/build/asan/bin/postgresql.jar:/tmp/dist-test-taskTVw2hL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.154.20:42415/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-07 18:15:01,088 [I] Table x_portal_user does not exist in database ranger
2026-08-07 18:15:01,089 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTVw2hL/build/asan/bin/postgresql.jar:/tmp/dist-test-taskTVw2hL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.154.20:42415/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-07 18:15:01,368 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-07 18:15:01,369 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTVw2hL/build/asan/bin/postgresql.jar:/tmp/dist-test-taskTVw2hL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.154.20:42415/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-07 18:15:01,651 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTVw2hL/build/asan/bin/postgresql.jar:/tmp/dist-test-taskTVw2hL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.154.20:42415/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-07 18:15:01,930 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-07 18:15:01,930 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTVw2hL/build/asan/bin/postgresql.jar:/tmp/dist-test-taskTVw2hL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.154.20:42415/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskTVw2hL/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-07 18:15:02.997 UTC [4034] WARNING: there is no transaction in progress
2026-08-07 18:15:03.017 UTC [4034] WARNING: there is no transaction in progress
2026-08-07 18:15:03.035 UTC [4034] WARNING: there is no transaction in progress
2026-08-07 18:15:03.052 UTC [4034] WARNING: there is no transaction in progress
2026-08-07 18:15:03.071 UTC [4034] WARNING: there is no transaction in progress
2026-08-07 18:15:03.093 UTC [4034] WARNING: there is no transaction in progress
2026-08-07 18:15:03.154 UTC [4034] WARNING: there is no transaction in progress
2026-08-07 18:15:03.162 UTC [4034] WARNING: there is no transaction in progress
2026-08-07 18:15:03.176 UTC [4034] WARNING: there is no transaction in progress
2026-08-07 18:15:03.187 UTC [4034] WARNING: there is no transaction in progress
2026-08-07 18:15:03.198 UTC [4034] WARNING: there is no transaction in progress
2026-08-07 18:15:03.214 UTC [4034] WARNING: there is no transaction in progress
2026-08-07 18:15:03.225 UTC [4034] WARNING: there is no transaction in progress
2026-08-07 18:15:03.232 UTC [4034] WARNING: there is no transaction in progress
2026-08-07 18:15:03.239 UTC [4034] WARNING: there is no transaction in progress
2026-08-07 18:15:03.246 UTC [4034] WARNING: there is no transaction in progress
2026-08-07 18:15:03,945 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-07 18:15:03,946 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTVw2hL/build/asan/bin/postgresql.jar:/tmp/dist-test-taskTVw2hL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.154.20:42415/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-07 18:15:04,215 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-07 18:15:04,215 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTVw2hL/build/asan/bin/postgresql.jar:/tmp/dist-test-taskTVw2hL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.154.20:42415/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-07 18:15:04,472 [I] Table x_portal_user already exists in database 'ranger'
2026-08-07 18:15:04,472 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTVw2hL/build/asan/bin/postgresql.jar:/tmp/dist-test-taskTVw2hL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.154.20:42415/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-07 18:15:04,758 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-07 18:15:04,758 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTVw2hL/build/asan/bin/postgresql.jar:/tmp/dist-test-taskTVw2hL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.154.20:42415/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-07 18:15:05,011 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTVw2hL/build/asan/bin/postgresql.jar:/tmp/dist-test-taskTVw2hL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.154.20:42415/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-07 18:15:05,285 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTVw2hL/build/asan/bin/postgresql.jar:/tmp/dist-test-taskTVw2hL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.154.20:42415/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-gkw7.c.gcp-upstream.internal';"
2026-08-07 18:15:05,551 [I] CORE_DB_SCHEMA import status has been updated
2026-08-07 18:15:05,552 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTVw2hL/build/asan/bin/postgresql.jar:/tmp/dist-test-taskTVw2hL/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.154.20:42415/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-07 18:15:05,823 [I] DB_PATCHES have already been applied
I20260807 18:15:05.829854 19048 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/minicluster-data/ranger-admin:/tmp/dist-test-taskTVw2hL/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskTVw2hL/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskTVw2hL/build/asan/bin/postgresql.jar:/tmp/dist-test-taskTVw2hL/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260807 18:15:05.829960 19048 mini_ranger.cc:193] Using host: 127.18.154.20
I20260807 18:15:05.851867 19048 mini_ranger.cc:227] Ranger admin URL: http://127.18.154.20:50901
Aug 07, 2026 6:15:06 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 07, 2026 6:15:06 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 07, 2026 6:15:06 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/minicluster-data/ranger-admin/ews/webapp
Aug 07, 2026 6:15:06 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 07, 2026 6:15:06 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 07, 2026 6:15:06 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/minicluster-data/ranger-admin/ews/webapp].
Aug 07, 2026 6:15:06 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260807 18:15:07.267623 19048 mini_ranger.cc:160] Time spent starting Ranger: real 9.487s user 0.003s sys 0.063s
Aug 07, 2026 6:15:28 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 07, 2026 6:15:28 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 07, 2026 6:15:28 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 07, 2026 6:15:28 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 07, 2026 6:15:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 07, 2026 6:15:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 07, 2026 6:15:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 07, 2026 6:15:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 07, 2026 6:15:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 07, 2026 6:15:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 07, 2026 6:15:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 07, 2026 6:15:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 07, 2026 6:15:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 07, 2026 6:15:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 07, 2026 6:15:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 07, 2026 6:15:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 07, 2026 6:15:28 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 07, 2026 6:15:29 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260807 18:15:29.913558 19048 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.18.154.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.18.154.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.18.154.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/krb5kdc/rangerkms_127.18.154.20@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.18.154.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/krb5kdc/rangerkms_127.18.154.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.18.154.20@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.18.154.20@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.18.154.20@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/krb5kdc/HTTP_127.18.154.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.18.154.20@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/krb5kdc/HTTP_127.18.154.20@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 07 18:15:30 dist-test-slave-gkw7 krb5kdc[3738](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126530, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260807 18:15:30.144330 19048 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260807 18:15:30.171856 19048 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260807 18:15:31.169103 19048 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260807 18:15:31.169268 19048 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/minicluster-data/ranger-kms
I20260807 18:15:31.169330 19048 mini_ranger_kms.cc:227] Using postgres at 127.18.154.20:42415
2026-08-07 18:15:31,547 [I] DB FLAVOR :POSTGRES
2026-08-07 18:15:31,547 [I] --------- Verifying Ranger DB connection ---------
2026-08-07 18:15:31,547 [I] Checking connection
2026-08-07 18:15:31,547 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTVw2hL/build/asan/bin/postgresql.jar:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.154.20:42415/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-07 18:15:31,808 [I] connection success
2026-08-07 18:15:31,808 [I] --------- Verifying Ranger DB tables ---------
2026-08-07 18:15:31,809 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-07 18:15:31,809 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTVw2hL/build/asan/bin/postgresql.jar:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.154.20:42415/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-07 18:15:32,057 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-07 18:15:32,057 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-07 18:15:32,057 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-07 18:15:32,057 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTVw2hL/build/asan/bin/postgresql.jar:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.154.20:42415/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-07 18:15:32,328 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260807 18:15:32.863481 19048 mini_ranger_kms.cc:310] Created kms service
I20260807 18:15:33.044246 19048 mini_ranger_kms.cc:326] Created kudu user
I20260807 18:15:33.092291 19048 mini_ranger_kms.cc:343] Created rangerkms user
Aug 07 18:15:33 dist-test-slave-gkw7 krb5kdc[3738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126530, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.18.154.20@KRBTEST.COM
I20260807 18:15:33.550351 19048 mini_ranger_kms.cc:384] Added ranger policy
I20260807 18:15:33.550717 19048 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/minicluster-data/ranger-kms:/tmp/dist-test-taskTVw2hL/build/asan/bin/postgresql.jar:/tmp/dist-test-taskTVw2hL/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskTVw2hL/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260807 18:15:33.550784 19048 mini_ranger_kms.cc:240] Using host: 127.18.154.20
I20260807 18:15:33.573307 19048 mini_ranger_kms.cc:276] Ranger KMS PID: 4294
I20260807 18:15:33.573457 19048 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.18.154.20:36567
Aug 07, 2026 6:15:34 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 07, 2026 6:15:34 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 07, 2026 6:15:34 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 07, 2026 6:15:34 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 07, 2026 6:15:35 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/minicluster-data/ranger-kms/ews/webapp].
Aug 07, 2026 6:15:35 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 07 18:15:35 dist-test-slave-gkw7 krb5kdc[3738](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786126535, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.18.154.20@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:15:36.262883 19048 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.119s user 0.077s sys 0.338s
I20260807 18:15:36.263182 19048 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260807 18:15:36.263391 19048 mini_ranger_kms.cc:401] 127.18.154.20:36567/kms/v1/keys
Aug 07 18:15:36 dist-test-slave-gkw7 krb5kdc[3738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126535, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.18.154.20@KRBTEST.COM for HTTP/127.18.154.20@KRBTEST.COM
Aug 07 18:15:36 dist-test-slave-gkw7 krb5kdc[3738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126535, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.18.154.20@KRBTEST.COM for HTTP/127.18.154.20@KRBTEST.COM
Aug 07 18:15:39 dist-test-slave-gkw7 krb5kdc[3738](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126539, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.18.154.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.154.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.154.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.62.keytab.
Entry for principal kudu/127.18.154.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.62.keytab.
I20260807 18:15:39.159176 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.154.62:40369
--webserver_interface=127.18.154.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.62.keytab
--principal=kudu/127.18.154.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.154.20:46707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.154.62:40369
--ranger_config_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.154.20:36567/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:15:39.512734 4365 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:15:39.513190 4365 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:15:39.513509 4365 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:15:39.521095 4365 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:15:39.521257 4365 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:15:39.521363 4365 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:15:39.521466 4365 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:15:39.521569 4365 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:15:39.531863 4365 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.154.20:46707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/minicluster-data/master-0/wal
--ranger_kms_url=127.18.154.20:36567/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.18.154.62:40369
--ranger_config_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.62.keytab
--principal=kudu/127.18.154.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.154.62:40369
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.154.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:15:39.533250 4365 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:15:39.534657 4365 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:15:39.547379 4371 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:15:39.547355 4372 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:15:39.547355 4375 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:15:39.548276 4365 server_base.cc:1034] running on GCE node
I20260807 18:15:39.549073 4365 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:15:39.550572 4365 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:15:39.551786 4365 hybrid_clock.cc:648] HybridClock initialized: now 1786126539551726 us; error 52 us; skew 500 ppm
I20260807 18:15:39.552160 4365 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:15:39 dist-test-slave-gkw7 krb5kdc[3738](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126539, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:15:39.561580 4365 init.cc:377] Logged in from keytab as kudu/127.18.154.62@KRBTEST.COM (short username kudu)
I20260807 18:15:39.563354 4365 webserver.cc:460] Webserver started at http://127.18.154.62:41113/ using document root <none> and password file <none>
I20260807 18:15:39.564075 4365 fs_manager.cc:359] Metadata directory not provided
I20260807 18:15:39.564226 4365 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:15:39.564554 4365 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 18:15:39 dist-test-slave-gkw7 krb5kdc[3738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126539, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.62@KRBTEST.COM for HTTP/127.18.154.20@KRBTEST.COM
I20260807 18:15:39.618479 4365 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/minicluster-data/master-0/data/instance:
uuid: "bf02f6e8e4fb4234877a7e85aa40706f"
format_stamp: "Formatted at 2026-08-07 18:15:39 on dist-test-slave-gkw7"
server_key: "6b60884763053f9bdf125575a66fdc80"
server_key_iv: "2bfcd29561e7b3de981f98e80112b09c"
server_key_version: "kuduclusterkey@0"
I20260807 18:15:39.619460 4365 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/minicluster-data/master-0/wal/instance:
uuid: "bf02f6e8e4fb4234877a7e85aa40706f"
format_stamp: "Formatted at 2026-08-07 18:15:39 on dist-test-slave-gkw7"
server_key: "6b60884763053f9bdf125575a66fdc80"
server_key_iv: "2bfcd29561e7b3de981f98e80112b09c"
server_key_version: "kuduclusterkey@0"
I20260807 18:15:39.625476 4365 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260807 18:15:39.643261 4382 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:15:39.645309 4365 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260807 18:15:39.645531 4365 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/minicluster-data/master-0/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/minicluster-data/master-0/wal
uuid: "bf02f6e8e4fb4234877a7e85aa40706f"
format_stamp: "Formatted at 2026-08-07 18:15:39 on dist-test-slave-gkw7"
server_key: "6b60884763053f9bdf125575a66fdc80"
server_key_iv: "2bfcd29561e7b3de981f98e80112b09c"
server_key_version: "kuduclusterkey@0"
I20260807 18:15:39.645839 4365 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:15:39.666036 4365 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:15:39.670050 4365 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:15:39.670423 4365 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:15:39.690572 4365 rpc_server.cc:307] RPC server started. Bound to: 127.18.154.62:40369
I20260807 18:15:39.690599 4457 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.154.62:40369 every 8 connection(s)
I20260807 18:15:39.692081 4365 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
I20260807 18:15:39.696151 4460 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:15:39.699800 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 4365
I20260807 18:15:39.700109 19048 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/minicluster-data/master-0/wal/instance
I20260807 18:15:39.701012 19048 external_mini_cluster.cc:1428] Setting key 414aa26d492f15b1f5387f5f8c45f6aa
I20260807 18:15:39.710505 4460 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bf02f6e8e4fb4234877a7e85aa40706f: Bootstrap starting.
I20260807 18:15:39.714887 4460 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bf02f6e8e4fb4234877a7e85aa40706f: Neither blocks nor log segments found. Creating new log.
I20260807 18:15:39.716357 4460 log.cc:824] T 00000000000000000000000000000000 P bf02f6e8e4fb4234877a7e85aa40706f: Log is configured to *not* fsync() on all Append() calls
Aug 07 18:15:39 dist-test-slave-gkw7 krb5kdc[3738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126539, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.154.62@KRBTEST.COM
I20260807 18:15:39.719481 4460 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bf02f6e8e4fb4234877a7e85aa40706f: No bootstrap required, opened a new log
I20260807 18:15:39.726713 4464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:39.703206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57954 (local address 127.18.154.62:40369)
0807 18:15:39.704075 (+ 869us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:39.704089 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:39.704133 (+ 44us) server_negotiation.cc:407] Connection header received
0807 18:15:39.705807 (+ 1674us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:39.705834 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:39.706251 (+ 417us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:39.706612 (+ 361us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:15:39.708506 (+ 1894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:39.710200 (+ 1694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:39.712183 (+ 1983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:39.712906 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:39.713049 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:39.718215 (+ 5166us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:15:39.718224 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:15:39.718241 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:15:39.718287 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:15:39.721708 (+ 3421us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:15:39.722732 (+ 1024us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:15:39.722742 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:15:39.722752 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:15:39.722993 (+ 241us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:15:39.723461 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:15:39.723467 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:15:39.723471 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:15:39.724182 (+ 711us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:15:39.724531 (+ 349us) server_negotiation.cc:1035] Waiting for connection context
0807 18:15:39.725088 (+ 557us) server_negotiation.cc:299] Negotiation successful
0807 18:15:39.725596 (+ 508us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":652,"thread_start_us":431,"threads_started":1}
I20260807 18:15:39.729297 4460 raft_consensus.cc:348] T 00000000000000000000000000000000 P bf02f6e8e4fb4234877a7e85aa40706f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf02f6e8e4fb4234877a7e85aa40706f" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 40369 } }
I20260807 18:15:39.729771 4460 raft_consensus.cc:374] T 00000000000000000000000000000000 P bf02f6e8e4fb4234877a7e85aa40706f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:15:39.729887 4460 raft_consensus.cc:729] T 00000000000000000000000000000000 P bf02f6e8e4fb4234877a7e85aa40706f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf02f6e8e4fb4234877a7e85aa40706f, State: Initialized, Role: FOLLOWER
I20260807 18:15:39.730449 4460 consensus_queue.cc:260] T 00000000000000000000000000000000 P bf02f6e8e4fb4234877a7e85aa40706f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf02f6e8e4fb4234877a7e85aa40706f" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 40369 } }
I20260807 18:15:39.730710 4460 raft_consensus.cc:388] T 00000000000000000000000000000000 P bf02f6e8e4fb4234877a7e85aa40706f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:15:39.730904 4460 raft_consensus.cc:482] T 00000000000000000000000000000000 P bf02f6e8e4fb4234877a7e85aa40706f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:15:39.731094 4460 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bf02f6e8e4fb4234877a7e85aa40706f [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:15:39.733261 4460 raft_consensus.cc:504] T 00000000000000000000000000000000 P bf02f6e8e4fb4234877a7e85aa40706f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf02f6e8e4fb4234877a7e85aa40706f" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 40369 } }
I20260807 18:15:39.733842 4460 leader_election.cc:302] T 00000000000000000000000000000000 P bf02f6e8e4fb4234877a7e85aa40706f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bf02f6e8e4fb4234877a7e85aa40706f; no voters:
I20260807 18:15:39.734431 4460 leader_election.cc:288] T 00000000000000000000000000000000 P bf02f6e8e4fb4234877a7e85aa40706f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:15:39.734591 4466 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bf02f6e8e4fb4234877a7e85aa40706f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:15:39.735699 4466 raft_consensus.cc:686] T 00000000000000000000000000000000 P bf02f6e8e4fb4234877a7e85aa40706f [term 1 LEADER]: Becoming Leader. State: Replica: bf02f6e8e4fb4234877a7e85aa40706f, State: Running, Role: LEADER
I20260807 18:15:39.736356 4466 consensus_queue.cc:237] T 00000000000000000000000000000000 P bf02f6e8e4fb4234877a7e85aa40706f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf02f6e8e4fb4234877a7e85aa40706f" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 40369 } }
I20260807 18:15:39.736738 4460 sys_catalog.cc:564] T 00000000000000000000000000000000 P bf02f6e8e4fb4234877a7e85aa40706f [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:15:39.739460 4471 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf02f6e8e4fb4234877a7e85aa40706f [sys.catalog]: SysCatalogTable state changed. Reason: New leader bf02f6e8e4fb4234877a7e85aa40706f. Latest consensus state: current_term: 1 leader_uuid: "bf02f6e8e4fb4234877a7e85aa40706f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf02f6e8e4fb4234877a7e85aa40706f" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 40369 } } }
I20260807 18:15:39.741888 4471 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf02f6e8e4fb4234877a7e85aa40706f [sys.catalog]: This master's current role is: LEADER
I20260807 18:15:39.739432 4469 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf02f6e8e4fb4234877a7e85aa40706f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bf02f6e8e4fb4234877a7e85aa40706f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf02f6e8e4fb4234877a7e85aa40706f" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 40369 } } }
I20260807 18:15:39.744872 4469 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf02f6e8e4fb4234877a7e85aa40706f [sys.catalog]: This master's current role is: LEADER
I20260807 18:15:39.746230 4483 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:15:39.751976 4483 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:15:39.752764 4460 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260807 18:15:39.781329 4483 catalog_manager.cc:1347] Generated new cluster ID: 9c5485dedb314f0a897c8ff4f8af5505
I20260807 18:15:39.781492 4483 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:15:39.794289 4483 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:15:39.796420 4483 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:15:39.809639 4483 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bf02f6e8e4fb4234877a7e85aa40706f: Generated new TSK 0
I20260807 18:15:39.810802 4483 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.154.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.154.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.154.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.1.keytab.
Entry for principal kudu/127.18.154.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.1.keytab.
I20260807 18:15:40.630204 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.154.20:36567/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.154.1:0
--local_ip_for_outbound_sockets=127.18.154.1
--webserver_interface=127.18.154.1
--webserver_port=0
--tserver_master_addrs=127.18.154.62:40369
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.1.keytab
--principal=kudu/127.18.154.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.154.20:46707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 18:15:40 dist-test-slave-gkw7 krb5kdc[3738](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786126540, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260807 18:15:41.110302 4534 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:15:41.110742 4534 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:15:41.110970 4534 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:15:41.122997 4534 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:15:41.123242 4534 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:15:41.123468 4534 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.154.1
I20260807 18:15:41.139669 4534 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.154.20:46707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.18.154.20:36567/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.1.keytab
--principal=kudu/127.18.154.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.154.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.154.1
--webserver_port=0
--tserver_master_addrs=127.18.154.62:40369
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.154.1
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:15:41.141512 4534 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:15:41.143385 4534 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:15:41.163021 4555 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:15:41.170071 4552 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:15:41.172508 4534 server_base.cc:1034] running on GCE node
W20260807 18:15:41.173570 4558 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:15:41.174561 4534 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:15:41.176298 4534 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:15:41.177657 4534 hybrid_clock.cc:648] HybridClock initialized: now 1786126541177543 us; error 104 us; skew 500 ppm
I20260807 18:15:41.178118 4534 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:15:41 dist-test-slave-gkw7 krb5kdc[3738](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126541, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:15:41.182613 4534 init.cc:377] Logged in from keytab as kudu/127.18.154.1@KRBTEST.COM (short username kudu)
I20260807 18:15:41.198411 4534 webserver.cc:460] Webserver started at http://127.18.154.1:45253/ using document root <none> and password file <none>
I20260807 18:15:41.199575 4534 fs_manager.cc:359] Metadata directory not provided
I20260807 18:15:41.199746 4534 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:15:41.200227 4534 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 18:15:41 dist-test-slave-gkw7 krb5kdc[3738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126541, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.1@KRBTEST.COM for HTTP/127.18.154.20@KRBTEST.COM
I20260807 18:15:41.223516 4534 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/minicluster-data/ts-0/data/instance:
uuid: "37214b71f71a4a17901e6cbafd0a44d2"
format_stamp: "Formatted at 2026-08-07 18:15:41 on dist-test-slave-gkw7"
server_key: "5cbcd9d0ffbf94aaeb988668cc8b364d"
server_key_iv: "d1752f70b221221a66ffbafe0d1cc120"
server_key_version: "kuduclusterkey@0"
I20260807 18:15:41.225170 4534 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/minicluster-data/ts-0/wal/instance:
uuid: "37214b71f71a4a17901e6cbafd0a44d2"
format_stamp: "Formatted at 2026-08-07 18:15:41 on dist-test-slave-gkw7"
server_key: "5cbcd9d0ffbf94aaeb988668cc8b364d"
server_key_iv: "d1752f70b221221a66ffbafe0d1cc120"
server_key_version: "kuduclusterkey@0"
I20260807 18:15:41.235931 4534 fs_manager.cc:691] Time spent creating directory manager: real 0.010s user 0.005s sys 0.003s
I20260807 18:15:41.279069 4568 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:15:41.283270 4534 fs_manager.cc:725] Time spent opening block manager: real 0.020s user 0.005s sys 0.009s
I20260807 18:15:41.283596 4534 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/minicluster-data/ts-0/wal
uuid: "37214b71f71a4a17901e6cbafd0a44d2"
format_stamp: "Formatted at 2026-08-07 18:15:41 on dist-test-slave-gkw7"
server_key: "5cbcd9d0ffbf94aaeb988668cc8b364d"
server_key_iv: "d1752f70b221221a66ffbafe0d1cc120"
server_key_version: "kuduclusterkey@0"
I20260807 18:15:41.283957 4534 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:15:41.337359 4534 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:15:41.343586 4534 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:15:41.343987 4534 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:15:41.345371 4534 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:15:41.347730 4534 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:15:41.347954 4534 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260807 18:15:41.348126 4534 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:15:41.348225 4534 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:15:41.413403 4534 rpc_server.cc:307] RPC server started. Bound to: 127.18.154.1:34641
I20260807 18:15:41.413446 4736 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.154.1:34641 every 8 connection(s)
I20260807 18:15:41.415200 4534 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/minicluster-data/ts-0/data/info.pb
I20260807 18:15:41.418234 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 4534
I20260807 18:15:41.418478 19048 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/minicluster-data/ts-0/wal/instance
I20260807 18:15:41.419628 19048 external_mini_cluster.cc:1428] Setting key 7696f3fad595be80c1b2ac42e6a11c67
Aug 07 18:15:41 dist-test-slave-gkw7 krb5kdc[3738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126541, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.1@KRBTEST.COM for kudu/127.18.154.62@KRBTEST.COM
Aug 07 18:15:41 dist-test-slave-gkw7 krb5kdc[3738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126540, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.62@KRBTEST.COM for HTTP/127.18.154.20@KRBTEST.COM
WARNING: no policy specified for kudu/127.18.154.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.154.2@KRBTEST.COM" created.
I20260807 18:15:41.486776 4739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:41.422509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.1:59809 (local address 127.18.154.62:40369)
0807 18:15:41.446224 (+ 23715us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:41.446235 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:41.446268 (+ 33us) server_negotiation.cc:407] Connection header received
0807 18:15:41.457748 (+ 11480us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:41.457758 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:41.457890 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:41.457986 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:15:41.460832 (+ 2846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:41.462178 (+ 1346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:41.465052 (+ 2874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:41.465893 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:41.466012 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:41.471131 (+ 5119us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:15:41.471140 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:15:41.471146 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:15:41.471207 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:15:41.475865 (+ 4658us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:15:41.476810 (+ 945us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:15:41.476818 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:15:41.476821 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:15:41.476967 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:15:41.477834 (+ 867us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:15:41.477840 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:15:41.477843 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:15:41.478232 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:15:41.478419 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0807 18:15:41.485899 (+ 7480us) server_negotiation.cc:299] Negotiation successful
0807 18:15:41.486231 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23510,"thread_start_us":24858,"threads_started":1}
I20260807 18:15:41.489831 4740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:41.423502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:40369 (local address 127.18.154.1:59809)
0807 18:15:41.444039 (+ 20537us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:41.444120 (+ 81us) client_negotiation.cc:174] Beginning negotiation
0807 18:15:41.453739 (+ 9619us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:15:41.458637 (+ 4898us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:15:41.458667 (+ 30us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:15:41.459475 (+ 808us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:15:41.460580 (+ 1105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:41.460615 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:41.462465 (+ 1850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:41.462473 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:41.464873 (+ 2400us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:41.464892 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:41.466398 (+ 1506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:41.466407 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:41.466752 (+ 345us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:41.467609 (+ 857us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:15:41.467662 (+ 53us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:15:41.470822 (+ 3160us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:15:41.476035 (+ 5213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:15:41.476044 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:15:41.476069 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:15:41.476651 (+ 582us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:15:41.477109 (+ 458us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:15:41.477118 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:15:41.477120 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:15:41.477343 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:15:41.478686 (+ 1343us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:15:41.478695 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:15:41.479223 (+ 528us) client_negotiation.cc:769] Sending connection context
0807 18:15:41.487364 (+ 8141us) client_negotiation.cc:240] Negotiation successful
0807 18:15:41.488104 (+ 740us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20016,"thread_start_us":21869,"threads_started":1}
I20260807 18:15:41.492563 4737 heartbeater.cc:347] Connected to a master server at 127.18.154.62:40369
I20260807 18:15:41.492956 4737 heartbeater.cc:464] Registering TS with master...
I20260807 18:15:41.494007 4737 heartbeater.cc:511] Master 127.18.154.62:40369 requested a full tablet report, sending...
I20260807 18:15:41.497157 4409 ts_manager.cc:194] Registered new tserver with Master: 37214b71f71a4a17901e6cbafd0a44d2 (127.18.154.1:34641)
I20260807 18:15:41.499655 4409 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.154.1@KRBTEST.COM'} at 127.18.154.1:59809
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.154.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.2.keytab.
Entry for principal kudu/127.18.154.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.2.keytab.
W20260807 18:15:41.525992 19048 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.18.154.2: real 0.101s user 0.003s sys 0.023s
I20260807 18:15:41.527208 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.154.20:36567/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.154.2:0
--local_ip_for_outbound_sockets=127.18.154.2
--webserver_interface=127.18.154.2
--webserver_port=0
--tserver_master_addrs=127.18.154.62:40369
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.2.keytab
--principal=kudu/127.18.154.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.154.20:46707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 18:15:41 dist-test-slave-gkw7 krb5kdc[3738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126540, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.62@KRBTEST.COM for HTTP/127.18.154.20@KRBTEST.COM
W20260807 18:15:42.100886 4745 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:15:42.101468 4745 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:15:42.101703 4745 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:15:42.110996 4745 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:15:42.111174 4745 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:15:42.111375 4745 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.154.2
I20260807 18:15:42.125950 4745 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.154.20:46707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.18.154.20:36567/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.2.keytab
--principal=kudu/127.18.154.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.154.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.154.2
--webserver_port=0
--tserver_master_addrs=127.18.154.62:40369
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.154.2
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:15:42.127851 4745 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:15:42.129729 4745 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:15:42.155952 4756 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:15:42.156708 4757 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:15:42.164255 4759 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:15:42.164785 4745 server_base.cc:1034] running on GCE node
I20260807 18:15:42.165815 4745 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:15:42.170288 4745 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:15:42.171654 4745 hybrid_clock.cc:648] HybridClock initialized: now 1786126542171587 us; error 64 us; skew 500 ppm
I20260807 18:15:42.172125 4745 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:15:42 dist-test-slave-gkw7 krb5kdc[3738](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126542, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:15:42.176579 4745 init.cc:377] Logged in from keytab as kudu/127.18.154.2@KRBTEST.COM (short username kudu)
I20260807 18:15:42.189347 4745 webserver.cc:460] Webserver started at http://127.18.154.2:44583/ using document root <none> and password file <none>
I20260807 18:15:42.190582 4745 fs_manager.cc:359] Metadata directory not provided
I20260807 18:15:42.190755 4745 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:15:42.191202 4745 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 18:15:42 dist-test-slave-gkw7 krb5kdc[3738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126542, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.2@KRBTEST.COM for HTTP/127.18.154.20@KRBTEST.COM
I20260807 18:15:42.209969 4745 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/minicluster-data/ts-1/data/instance:
uuid: "ae903e78a8a44c1c9fd3629bcb34d58a"
format_stamp: "Formatted at 2026-08-07 18:15:42 on dist-test-slave-gkw7"
server_key: "af0b47888b5df4ace1ff43766c77cccc"
server_key_iv: "355f6c22f1121e7f5f35bbaeac661692"
server_key_version: "kuduclusterkey@0"
I20260807 18:15:42.211205 4745 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/minicluster-data/ts-1/wal/instance:
uuid: "ae903e78a8a44c1c9fd3629bcb34d58a"
format_stamp: "Formatted at 2026-08-07 18:15:42 on dist-test-slave-gkw7"
server_key: "af0b47888b5df4ace1ff43766c77cccc"
server_key_iv: "355f6c22f1121e7f5f35bbaeac661692"
server_key_version: "kuduclusterkey@0"
I20260807 18:15:42.224777 4745 fs_manager.cc:691] Time spent creating directory manager: real 0.013s user 0.009s sys 0.004s
I20260807 18:15:42.255592 4770 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:15:42.258554 4745 fs_manager.cc:725] Time spent opening block manager: real 0.012s user 0.007s sys 0.000s
I20260807 18:15:42.258886 4745 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/minicluster-data/ts-1/wal
uuid: "ae903e78a8a44c1c9fd3629bcb34d58a"
format_stamp: "Formatted at 2026-08-07 18:15:42 on dist-test-slave-gkw7"
server_key: "af0b47888b5df4ace1ff43766c77cccc"
server_key_iv: "355f6c22f1121e7f5f35bbaeac661692"
server_key_version: "kuduclusterkey@0"
I20260807 18:15:42.259207 4745 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:15:42.285190 4745 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:15:42.291003 4745 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:15:42.291358 4745 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:15:42.292670 4745 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:15:42.294672 4745 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:15:42.294772 4745 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:15:42.294905 4745 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:15:42.294970 4745 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:15:42.354449 4745 rpc_server.cc:307] RPC server started. Bound to: 127.18.154.2:44659
I20260807 18:15:42.354501 4936 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.154.2:44659 every 8 connection(s)
I20260807 18:15:42.356299 4745 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/minicluster-data/ts-1/data/info.pb
I20260807 18:15:42.364552 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 4745
I20260807 18:15:42.364748 19048 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/minicluster-data/ts-1/wal/instance
I20260807 18:15:42.365327 19048 external_mini_cluster.cc:1428] Setting key 85216da2a177de86cbd5695c465de6e6
Aug 07 18:15:42 dist-test-slave-gkw7 krb5kdc[3738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126542, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.2@KRBTEST.COM for kudu/127.18.154.62@KRBTEST.COM
I20260807 18:15:42.395816 4940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:42.362120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.2:38765 (local address 127.18.154.62:40369)
0807 18:15:42.375097 (+ 12977us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:42.375107 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:42.375135 (+ 28us) server_negotiation.cc:407] Connection header received
0807 18:15:42.375219 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:42.375228 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:42.375340 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:42.375483 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:15:42.379795 (+ 4312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:42.380597 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:42.382987 (+ 2390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:42.383418 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:42.383580 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:42.388165 (+ 4585us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:15:42.388170 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:15:42.388174 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:15:42.388207 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:15:42.391783 (+ 3576us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:15:42.392743 (+ 960us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:15:42.392749 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:15:42.392753 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:15:42.392913 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:15:42.393507 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:15:42.393512 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:15:42.393516 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:15:42.393943 (+ 427us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:15:42.394094 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0807 18:15:42.395138 (+ 1044us) server_negotiation.cc:299] Negotiation successful
0807 18:15:42.395388 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12780,"thread_start_us":12607,"threads_started":1}
I20260807 18:15:42.398455 4939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:42.361452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:40369 (local address 127.18.154.2:38765)
0807 18:15:42.370381 (+ 8929us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:42.370459 (+ 78us) client_negotiation.cc:174] Beginning negotiation
0807 18:15:42.374085 (+ 3626us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:15:42.377986 (+ 3901us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:15:42.378000 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:15:42.378707 (+ 707us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:15:42.379579 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:42.379608 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:42.380790 (+ 1182us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:42.380795 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:42.382835 (+ 2040us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:42.382852 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:42.383699 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:42.383706 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:42.383989 (+ 283us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:42.384850 (+ 861us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:15:42.384881 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:15:42.387902 (+ 3021us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:15:42.391962 (+ 4060us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:15:42.391972 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:15:42.391991 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:15:42.392597 (+ 606us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:15:42.393074 (+ 477us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:15:42.393079 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:15:42.393083 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:15:42.393372 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:15:42.394201 (+ 829us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:15:42.394210 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:15:42.394676 (+ 466us) client_negotiation.cc:769] Sending connection context
0807 18:15:42.396359 (+ 1683us) client_negotiation.cc:240] Negotiation successful
0807 18:15:42.396961 (+ 602us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8455,"thread_start_us":11919,"threads_started":1}
I20260807 18:15:42.400611 4937 heartbeater.cc:347] Connected to a master server at 127.18.154.62:40369
I20260807 18:15:42.400954 4937 heartbeater.cc:464] Registering TS with master...
I20260807 18:15:42.401928 4937 heartbeater.cc:511] Master 127.18.154.62:40369 requested a full tablet report, sending...
I20260807 18:15:42.403695 4409 ts_manager.cc:194] Registered new tserver with Master: ae903e78a8a44c1c9fd3629bcb34d58a (127.18.154.2:44659)
I20260807 18:15:42.404933 4409 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.154.2@KRBTEST.COM'} at 127.18.154.2:38765
WARNING: no policy specified for kudu/127.18.154.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.154.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.154.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.3.keytab.
Entry for principal kudu/127.18.154.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.3.keytab.
I20260807 18:15:42.446831 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.154.20:36567/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.154.3:0
--local_ip_for_outbound_sockets=127.18.154.3
--webserver_interface=127.18.154.3
--webserver_port=0
--tserver_master_addrs=127.18.154.62:40369
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.3.keytab
--principal=kudu/127.18.154.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.154.20:46707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 18:15:42.504528 4737 heartbeater.cc:503] Master 127.18.154.62:40369 was elected leader, sending a full tablet report...
W20260807 18:15:42.809278 4944 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:15:42.809670 4944 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:15:42.809859 4944 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:15:42.817353 4944 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:15:42.817518 4944 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:15:42.817757 4944 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.154.3
I20260807 18:15:42.828121 4944 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.154.20:46707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.18.154.20:36567/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.3.keytab
--principal=kudu/127.18.154.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.154.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.154.3
--webserver_port=0
--tserver_master_addrs=127.18.154.62:40369
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.154.3
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:15:42.829618 4944 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:15:42.831007 4944 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:15:42.845223 4951 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:15:42.845299 4953 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:15:42.845340 4950 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:15:42.846505 4944 server_base.cc:1034] running on GCE node
I20260807 18:15:42.847220 4944 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:15:42.848363 4944 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:15:42.849615 4944 hybrid_clock.cc:648] HybridClock initialized: now 1786126542849550 us; error 41 us; skew 500 ppm
I20260807 18:15:42.850003 4944 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:15:42 dist-test-slave-gkw7 krb5kdc[3738](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126542, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:15:42.853415 4944 init.cc:377] Logged in from keytab as kudu/127.18.154.3@KRBTEST.COM (short username kudu)
I20260807 18:15:42.855022 4944 webserver.cc:460] Webserver started at http://127.18.154.3:40275/ using document root <none> and password file <none>
I20260807 18:15:42.855715 4944 fs_manager.cc:359] Metadata directory not provided
I20260807 18:15:42.855875 4944 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:15:42.856190 4944 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 18:15:42 dist-test-slave-gkw7 krb5kdc[3738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126542, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.3@KRBTEST.COM for HTTP/127.18.154.20@KRBTEST.COM
I20260807 18:15:42.874874 4944 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/minicluster-data/ts-2/data/instance:
uuid: "42ed1c0b90f544d29feabe07cea7528d"
format_stamp: "Formatted at 2026-08-07 18:15:42 on dist-test-slave-gkw7"
server_key: "e274a96d2c9cb23befee18c1929ba6de"
server_key_iv: "f4e66858071297c7efb7b79bbf2e4ea3"
server_key_version: "kuduclusterkey@0"
I20260807 18:15:42.876147 4944 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/minicluster-data/ts-2/wal/instance:
uuid: "42ed1c0b90f544d29feabe07cea7528d"
format_stamp: "Formatted at 2026-08-07 18:15:42 on dist-test-slave-gkw7"
server_key: "e274a96d2c9cb23befee18c1929ba6de"
server_key_iv: "f4e66858071297c7efb7b79bbf2e4ea3"
server_key_version: "kuduclusterkey@0"
I20260807 18:15:42.883845 4944 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.009s sys 0.000s
I20260807 18:15:42.900430 4961 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:15:42.902100 4944 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 18:15:42.902305 4944 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/minicluster-data/ts-2/wal
uuid: "42ed1c0b90f544d29feabe07cea7528d"
format_stamp: "Formatted at 2026-08-07 18:15:42 on dist-test-slave-gkw7"
server_key: "e274a96d2c9cb23befee18c1929ba6de"
server_key_iv: "f4e66858071297c7efb7b79bbf2e4ea3"
server_key_version: "kuduclusterkey@0"
I20260807 18:15:42.902586 4944 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:15:42.920186 4944 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:15:42.924144 4944 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:15:42.924471 4944 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:15:42.925472 4944 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:15:42.926934 4944 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:15:42.927048 4944 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:15:42.927209 4944 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:15:42.927289 4944 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:15:42.962188 4944 rpc_server.cc:307] RPC server started. Bound to: 127.18.154.3:44187
I20260807 18:15:42.962231 5114 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.154.3:44187 every 8 connection(s)
I20260807 18:15:42.963554 4944 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/minicluster-data/ts-2/data/info.pb
I20260807 18:15:42.968844 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 4944
I20260807 18:15:42.969023 19048 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/minicluster-data/ts-2/wal/instance
I20260807 18:15:42.969605 19048 external_mini_cluster.cc:1428] Setting key c85e834706b69811c5c432ebb8b18cf4
Aug 07 18:15:42 dist-test-slave-gkw7 krb5kdc[3738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126542, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.3@KRBTEST.COM for kudu/127.18.154.62@KRBTEST.COM
I20260807 18:15:42.985087 5119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:42.966334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.3:41725 (local address 127.18.154.62:40369)
0807 18:15:42.966806 (+ 472us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:42.966811 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:42.967748 (+ 937us) server_negotiation.cc:407] Connection header received
0807 18:15:42.969949 (+ 2201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:42.969955 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:42.970061 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:42.970190 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:15:42.971670 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:42.972448 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:42.974240 (+ 1792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:42.974715 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:42.974868 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:42.978610 (+ 3742us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:15:42.978616 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:15:42.978621 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:15:42.978655 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:15:42.981460 (+ 2805us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:15:42.982402 (+ 942us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:15:42.982406 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:15:42.982409 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:15:42.982565 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:15:42.983037 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:15:42.983047 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:15:42.983050 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:15:42.983340 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:15:42.983546 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0807 18:15:42.984430 (+ 884us) server_negotiation.cc:299] Negotiation successful
0807 18:15:42.984688 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":365,"thread_start_us":238,"threads_started":1}
I20260807 18:15:42.986150 5120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:42.966881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:40369 (local address 127.18.154.3:41725)
0807 18:15:42.967567 (+ 686us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:42.967615 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 18:15:42.969610 (+ 1995us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:15:42.970454 (+ 844us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:15:42.970463 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:15:42.970881 (+ 418us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:15:42.971479 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:42.971497 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:42.972623 (+ 1126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:42.972628 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:42.974073 (+ 1445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:42.974095 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:42.974969 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:42.974980 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:42.975244 (+ 264us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:42.975951 (+ 707us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:15:42.975978 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:15:42.978336 (+ 2358us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:15:42.981683 (+ 3347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:15:42.981691 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:15:42.981704 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:15:42.982207 (+ 503us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:15:42.982680 (+ 473us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:15:42.982683 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:15:42.982685 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:15:42.982895 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:15:42.983519 (+ 624us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:15:42.983526 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:15:42.983856 (+ 330us) client_negotiation.cc:769] Sending connection context
0807 18:15:42.984254 (+ 398us) client_negotiation.cc:240] Negotiation successful
0807 18:15:42.984788 (+ 534us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":453,"thread_start_us":274,"threads_started":1}
I20260807 18:15:42.987617 5115 heartbeater.cc:347] Connected to a master server at 127.18.154.62:40369
I20260807 18:15:42.987964 5115 heartbeater.cc:464] Registering TS with master...
I20260807 18:15:42.988771 5115 heartbeater.cc:511] Master 127.18.154.62:40369 requested a full tablet report, sending...
I20260807 18:15:42.990477 4409 ts_manager.cc:194] Registered new tserver with Master: 42ed1c0b90f544d29feabe07cea7528d (127.18.154.3:44187)
I20260807 18:15:42.991317 4409 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.154.3@KRBTEST.COM'} at 127.18.154.3:41725
I20260807 18:15:42.998327 19048 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 18:15:42.999992 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 4534
Aug 07 18:15:43 dist-test-slave-gkw7 krb5kdc[3738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126539, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.154.1@KRBTEST.COM
I20260807 18:15:43.019258 5122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:43.001154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42018 (local address 127.18.154.1:34641)
0807 18:15:43.002167 (+ 1013us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:43.002178 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:43.002210 (+ 32us) server_negotiation.cc:407] Connection header received
0807 18:15:43.002283 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:43.002292 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:43.002544 (+ 252us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:43.002675 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:15:43.004249 (+ 1574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:43.005310 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:43.007010 (+ 1700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:43.007775 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:43.007882 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:43.012447 (+ 4565us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:15:43.012461 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:15:43.012477 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:15:43.012515 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:15:43.015645 (+ 3130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:15:43.016560 (+ 915us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:15:43.016570 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:15:43.016572 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:15:43.016735 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:15:43.017278 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:15:43.017282 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:15:43.017284 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:15:43.017782 (+ 498us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:15:43.018035 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0807 18:15:43.018574 (+ 539us) server_negotiation.cc:299] Negotiation successful
0807 18:15:43.018889 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":783,"thread_start_us":433,"threads_started":1}
I20260807 18:15:43.020754 4729 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:15:43.105703 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 4534
I20260807 18:15:43.130658 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 4745
Aug 07 18:15:43 dist-test-slave-gkw7 krb5kdc[3738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126539, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.154.2@KRBTEST.COM
I20260807 18:15:43.152999 5126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:43.131688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45734 (local address 127.18.154.2:44659)
0807 18:15:43.132247 (+ 559us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:43.132257 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:43.132294 (+ 37us) server_negotiation.cc:407] Connection header received
0807 18:15:43.132357 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:43.132363 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:43.132571 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:43.132731 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:15:43.134053 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:43.135077 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:43.136536 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:43.137298 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:43.137400 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:43.146480 (+ 9080us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:15:43.146496 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:15:43.146513 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:15:43.146570 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:15:43.149977 (+ 3407us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:15:43.150782 (+ 805us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:15:43.150790 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:15:43.150793 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:15:43.150940 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:15:43.151408 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:15:43.151412 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:15:43.151414 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:15:43.151792 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:15:43.152044 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0807 18:15:43.152448 (+ 404us) server_negotiation.cc:299] Negotiation successful
0807 18:15:43.152680 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":398,"thread_start_us":298,"threads_started":1}
I20260807 18:15:43.154493 4930 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:15:43.238168 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 4745
I20260807 18:15:43.262624 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 4944
Aug 07 18:15:43 dist-test-slave-gkw7 krb5kdc[3738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126539, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.154.3@KRBTEST.COM
I20260807 18:15:43.282922 5130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:43.263884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53406 (local address 127.18.154.3:44187)
0807 18:15:43.264552 (+ 668us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:43.264559 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:43.264591 (+ 32us) server_negotiation.cc:407] Connection header received
0807 18:15:43.264669 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:43.264676 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:43.264903 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:43.265031 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:15:43.266675 (+ 1644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:43.267708 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:43.269227 (+ 1519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:43.269931 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:43.270033 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:43.275378 (+ 5345us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:15:43.275400 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:15:43.275426 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:15:43.275493 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:15:43.280018 (+ 4525us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:15:43.280933 (+ 915us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:15:43.280941 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:15:43.280943 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:15:43.281127 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:15:43.281462 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:15:43.281465 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:15:43.281467 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:15:43.281934 (+ 467us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:15:43.282245 (+ 311us) server_negotiation.cc:1035] Waiting for connection context
0807 18:15:43.282418 (+ 173us) server_negotiation.cc:299] Negotiation successful
0807 18:15:43.282624 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":517,"thread_start_us":352,"threads_started":1}
I20260807 18:15:43.284291 5110 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:15:43.369334 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 4944
I20260807 18:15:43.391893 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 4365
I20260807 18:15:43.393388 4450 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:15:43.504920 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 4365
I20260807 18:15:43.540594 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.154.62:40369
--webserver_interface=127.18.154.62
--webserver_port=41113
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.62.keytab
--principal=kudu/127.18.154.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.154.20:46707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.154.62:40369
--ranger_config_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.154.20:36567/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.18.154.20:36567/kms with env {KRB5CCNAME=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:15:43.897174 5134 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:15:43.897500 5134 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:15:43.897706 5134 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:15:43.905248 5134 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:15:43.905366 5134 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:15:43.905423 5134 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:15:43.905489 5134 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:15:43.905601 5134 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:15:43.916208 5134 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.154.20:46707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.18.154.20:36567/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.18.154.62:40369
--ranger_config_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.62.keytab
--principal=kudu/127.18.154.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.154.62:40369
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.154.62
--webserver_port=41113
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:15:43.917724 5134 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:15:43.919131 5134 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:15:43.932086 5140 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:15:43.932070 5141 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:15:43.932101 5143 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:15:43.933120 5134 server_base.cc:1034] running on GCE node
I20260807 18:15:43.934046 5134 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:15:43.935550 5134 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:15:43.936800 5134 hybrid_clock.cc:648] HybridClock initialized: now 1786126543936740 us; error 60 us; skew 500 ppm
I20260807 18:15:43.937232 5134 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:15:43 dist-test-slave-gkw7 krb5kdc[3738](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126543, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:15:43.940660 5134 init.cc:377] Logged in from keytab as kudu/127.18.154.62@KRBTEST.COM (short username kudu)
I20260807 18:15:43.942332 5134 webserver.cc:460] Webserver started at http://127.18.154.62:41113/ using document root <none> and password file <none>
I20260807 18:15:43.942973 5134 fs_manager.cc:359] Metadata directory not provided
I20260807 18:15:43.943084 5134 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 07 18:15:44 dist-test-slave-gkw7 krb5kdc[3738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126543, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.62@KRBTEST.COM for HTTP/127.18.154.20@KRBTEST.COM
I20260807 18:15:44.025831 5134 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 18:15:44.029641 5156 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:15:44.031370 5134 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260807 18:15:44.031594 5134 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/minicluster-data/master-0/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/minicluster-data/master-0/wal
uuid: "bf02f6e8e4fb4234877a7e85aa40706f"
format_stamp: "Formatted at 2026-08-07 18:15:39 on dist-test-slave-gkw7"
server_key: "6b60884763053f9bdf125575a66fdc80"
server_key_iv: "2bfcd29561e7b3de981f98e80112b09c"
server_key_version: "kuduclusterkey@0"
I20260807 18:15:44.032644 5134 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:15:44.053467 5134 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:15:44.057823 5134 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:15:44.058142 5134 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:15:44.078603 5134 rpc_server.cc:307] RPC server started. Bound to: 127.18.154.62:40369
I20260807 18:15:44.078653 5223 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.154.62:40369 every 8 connection(s)
I20260807 18:15:44.080385 5134 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
I20260807 18:15:44.088300 5224 sys_catalog.cc:263] Verifying existing consensus state
I20260807 18:15:44.090425 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 5134
I20260807 18:15:44.091331 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.154.20:36567/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.154.1:34641
--local_ip_for_outbound_sockets=127.18.154.1
--tserver_master_addrs=127.18.154.62:40369
--webserver_port=45253
--webserver_interface=127.18.154.1
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.1.keytab
--principal=kudu/127.18.154.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.154.20:46707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.18.154.20:36567/kms with env {KRB5CCNAME=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 18:15:44.092257 5224 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bf02f6e8e4fb4234877a7e85aa40706f: Bootstrap starting.
I20260807 18:15:44.113902 5224 log.cc:824] T 00000000000000000000000000000000 P bf02f6e8e4fb4234877a7e85aa40706f: Log is configured to *not* fsync() on all Append() calls
I20260807 18:15:44.124424 5224 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bf02f6e8e4fb4234877a7e85aa40706f: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 18:15:44.125087 5224 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bf02f6e8e4fb4234877a7e85aa40706f: Bootstrap complete.
I20260807 18:15:44.135772 5224 raft_consensus.cc:348] T 00000000000000000000000000000000 P bf02f6e8e4fb4234877a7e85aa40706f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf02f6e8e4fb4234877a7e85aa40706f" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 40369 } }
I20260807 18:15:44.136907 5224 raft_consensus.cc:729] T 00000000000000000000000000000000 P bf02f6e8e4fb4234877a7e85aa40706f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf02f6e8e4fb4234877a7e85aa40706f, State: Initialized, Role: FOLLOWER
I20260807 18:15:44.137774 5224 consensus_queue.cc:260] T 00000000000000000000000000000000 P bf02f6e8e4fb4234877a7e85aa40706f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf02f6e8e4fb4234877a7e85aa40706f" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 40369 } }
I20260807 18:15:44.138072 5224 raft_consensus.cc:388] T 00000000000000000000000000000000 P bf02f6e8e4fb4234877a7e85aa40706f [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:15:44.138235 5224 raft_consensus.cc:482] T 00000000000000000000000000000000 P bf02f6e8e4fb4234877a7e85aa40706f [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:15:44.138456 5224 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bf02f6e8e4fb4234877a7e85aa40706f [term 1 FOLLOWER]: Advancing to term 2
I20260807 18:15:44.143165 5224 raft_consensus.cc:504] T 00000000000000000000000000000000 P bf02f6e8e4fb4234877a7e85aa40706f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf02f6e8e4fb4234877a7e85aa40706f" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 40369 } }
I20260807 18:15:44.143747 5224 leader_election.cc:302] T 00000000000000000000000000000000 P bf02f6e8e4fb4234877a7e85aa40706f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bf02f6e8e4fb4234877a7e85aa40706f; no voters:
I20260807 18:15:44.144495 5224 leader_election.cc:288] T 00000000000000000000000000000000 P bf02f6e8e4fb4234877a7e85aa40706f [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 18:15:44.144649 5230 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bf02f6e8e4fb4234877a7e85aa40706f [term 2 FOLLOWER]: Leader election won for term 2
I20260807 18:15:44.145882 5230 raft_consensus.cc:686] T 00000000000000000000000000000000 P bf02f6e8e4fb4234877a7e85aa40706f [term 2 LEADER]: Becoming Leader. State: Replica: bf02f6e8e4fb4234877a7e85aa40706f, State: Running, Role: LEADER
I20260807 18:15:44.146503 5230 consensus_queue.cc:237] T 00000000000000000000000000000000 P bf02f6e8e4fb4234877a7e85aa40706f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf02f6e8e4fb4234877a7e85aa40706f" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 40369 } }
I20260807 18:15:44.147086 5224 sys_catalog.cc:564] T 00000000000000000000000000000000 P bf02f6e8e4fb4234877a7e85aa40706f [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:15:44.149461 5233 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf02f6e8e4fb4234877a7e85aa40706f [sys.catalog]: SysCatalogTable state changed. Reason: New leader bf02f6e8e4fb4234877a7e85aa40706f. Latest consensus state: current_term: 2 leader_uuid: "bf02f6e8e4fb4234877a7e85aa40706f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf02f6e8e4fb4234877a7e85aa40706f" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 40369 } } }
I20260807 18:15:44.149593 5231 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf02f6e8e4fb4234877a7e85aa40706f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "bf02f6e8e4fb4234877a7e85aa40706f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf02f6e8e4fb4234877a7e85aa40706f" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 40369 } } }
I20260807 18:15:44.150092 5231 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf02f6e8e4fb4234877a7e85aa40706f [sys.catalog]: This master's current role is: LEADER
I20260807 18:15:44.149955 5233 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf02f6e8e4fb4234877a7e85aa40706f [sys.catalog]: This master's current role is: LEADER
I20260807 18:15:44.151230 5237 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:15:44.161638 5237 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:15:44.163360 5237 catalog_manager.cc:1259] Loaded cluster ID: 9c5485dedb314f0a897c8ff4f8af5505
I20260807 18:15:44.163552 5237 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:15:44.167690 5237 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:15:44.169215 5237 catalog_manager.cc:5879] T 00000000000000000000000000000000 P bf02f6e8e4fb4234877a7e85aa40706f: Loaded TSK: 0
I20260807 18:15:44.172111 5237 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:15:44.177961 5224 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
W20260807 18:15:44.565492 5226 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:15:44.565997 5226 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:15:44.566210 5226 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:15:44.578598 5226 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:15:44.578768 5226 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:15:44.578979 5226 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.154.1
I20260807 18:15:44.595193 5226 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.154.20:46707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.18.154.20:36567/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.1.keytab
--principal=kudu/127.18.154.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.154.1:34641
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.154.1
--webserver_port=45253
--tserver_master_addrs=127.18.154.62:40369
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.154.1
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:15:44.597110 5226 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:15:44.598960 5226 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:15:44.635958 5273 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:15:44.636555 5275 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:15:44.642824 5279 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:15:44.644299 5226 server_base.cc:1034] running on GCE node
I20260807 18:15:44.645308 5226 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:15:44.647413 5226 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:15:44.663249 5226 hybrid_clock.cc:648] HybridClock initialized: now 1786126544663137 us; error 80 us; skew 500 ppm
I20260807 18:15:44.663858 5226 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:15:44 dist-test-slave-gkw7 krb5kdc[3738](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126544, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:15:44.673857 5226 init.cc:377] Logged in from keytab as kudu/127.18.154.1@KRBTEST.COM (short username kudu)
I20260807 18:15:44.677740 5226 webserver.cc:460] Webserver started at http://127.18.154.1:45253/ using document root <none> and password file <none>
I20260807 18:15:44.678861 5226 fs_manager.cc:359] Metadata directory not provided
I20260807 18:15:44.679052 5226 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 07 18:15:44 dist-test-slave-gkw7 krb5kdc[3738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126544, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.1@KRBTEST.COM for HTTP/127.18.154.20@KRBTEST.COM
I20260807 18:15:44.737128 5226 fs_manager.cc:709] Time spent opening directory manager: real 0.015s user 0.008s sys 0.006s
I20260807 18:15:44.748073 5296 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:15:44.752394 5226 fs_manager.cc:725] Time spent opening block manager: real 0.013s user 0.000s sys 0.004s
I20260807 18:15:44.752672 5226 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/minicluster-data/ts-0/wal
uuid: "37214b71f71a4a17901e6cbafd0a44d2"
format_stamp: "Formatted at 2026-08-07 18:15:41 on dist-test-slave-gkw7"
server_key: "5cbcd9d0ffbf94aaeb988668cc8b364d"
server_key_iv: "d1752f70b221221a66ffbafe0d1cc120"
server_key_version: "kuduclusterkey@0"
I20260807 18:15:44.754092 5226 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:15:44.789426 5226 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:15:44.795477 5226 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:15:44.795832 5226 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:15:44.799293 5226 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:15:44.801381 5226 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:15:44.801512 5226 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:15:44.801673 5226 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:15:44.801806 5226 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:15:44.878885 5226 rpc_server.cc:307] RPC server started. Bound to: 127.18.154.1:34641
I20260807 18:15:44.878964 5456 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.154.1:34641 every 8 connection(s)
I20260807 18:15:44.880848 5226 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/minicluster-data/ts-0/data/info.pb
I20260807 18:15:44.890271 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 5226
I20260807 18:15:44.891151 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.154.20:36567/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.154.2:44659
--local_ip_for_outbound_sockets=127.18.154.2
--tserver_master_addrs=127.18.154.62:40369
--webserver_port=44583
--webserver_interface=127.18.154.2
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.2.keytab
--principal=kudu/127.18.154.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.154.20:46707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.18.154.20:36567/kms with env {KRB5CCNAME=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 18:15:44 dist-test-slave-gkw7 krb5kdc[3738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126544, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.1@KRBTEST.COM for kudu/127.18.154.62@KRBTEST.COM
I20260807 18:15:44.933665 5460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:44.884749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:40369 (local address 127.18.154.1:37157)
0807 18:15:44.900847 (+ 16098us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:44.900924 (+ 77us) client_negotiation.cc:174] Beginning negotiation
0807 18:15:44.903857 (+ 2933us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:15:44.906306 (+ 2449us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:15:44.906317 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:15:44.907027 (+ 710us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:15:44.907923 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:44.907967 (+ 44us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:44.910913 (+ 2946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:44.910923 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:44.913549 (+ 2626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:44.913582 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:44.915155 (+ 1573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:44.915166 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:44.915559 (+ 393us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:44.916521 (+ 962us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:15:44.916561 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:15:44.919973 (+ 3412us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:15:44.927541 (+ 7568us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:15:44.927554 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:15:44.927575 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:15:44.928415 (+ 840us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:15:44.929247 (+ 832us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:15:44.929253 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:15:44.929258 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:15:44.929514 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:15:44.930682 (+ 1168us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:15:44.930692 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:15:44.931144 (+ 452us) client_negotiation.cc:769] Sending connection context
0807 18:15:44.931460 (+ 316us) client_negotiation.cc:240] Negotiation successful
0807 18:15:44.932106 (+ 646us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15725,"thread_start_us":16679,"threads_started":1}
I20260807 18:15:44.934123 5459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:44.884658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.1:37157 (local address 127.18.154.62:40369)
0807 18:15:44.899888 (+ 15230us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:44.899913 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:44.904378 (+ 4465us) server_negotiation.cc:407] Connection header received
0807 18:15:44.904981 (+ 603us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:44.905007 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:44.905663 (+ 656us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:44.906110 (+ 447us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:15:44.908232 (+ 2122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:44.910673 (+ 2441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:44.913759 (+ 3086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:44.914847 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:44.915068 (+ 221us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:44.920397 (+ 5329us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:15:44.920412 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:15:44.920440 (+ 28us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:15:44.920510 (+ 70us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:15:44.927283 (+ 6773us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:15:44.928833 (+ 1550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:15:44.928845 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:15:44.928855 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:15:44.929087 (+ 232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:15:44.929717 (+ 630us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:15:44.929723 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:15:44.929727 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:15:44.930489 (+ 762us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:15:44.931957 (+ 1468us) server_negotiation.cc:1035] Waiting for connection context
0807 18:15:44.932290 (+ 333us) server_negotiation.cc:299] Negotiation successful
0807 18:15:44.932750 (+ 460us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14817,"thread_start_us":15132,"threads_started":1}
I20260807 18:15:44.938086 5457 heartbeater.cc:347] Connected to a master server at 127.18.154.62:40369
I20260807 18:15:44.938445 5457 heartbeater.cc:464] Registering TS with master...
I20260807 18:15:44.939429 5457 heartbeater.cc:511] Master 127.18.154.62:40369 requested a full tablet report, sending...
W20260807 18:15:44.943347 5457 heartbeater.cc:644] Failed to heartbeat to 127.18.154.62:40369 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260807 18:15:44.943621 5457 heartbeater.cc:464] Registering TS with master...
I20260807 18:15:44.943962 5457 heartbeater.cc:511] Master 127.18.154.62:40369 requested a full tablet report, sending...
I20260807 18:15:44.945367 5457 heartbeater.cc:464] Registering TS with master...
I20260807 18:15:44.945693 5457 heartbeater.cc:511] Master 127.18.154.62:40369 requested a full tablet report, sending...
W20260807 18:15:44.946918 5457 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 18:15:45.366192 5463 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:15:45.366586 5463 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:15:45.366736 5463 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:15:45.378542 5463 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:15:45.378755 5463 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:15:45.378969 5463 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.154.2
I20260807 18:15:45.395052 5463 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.154.20:46707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.18.154.20:36567/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.2.keytab
--principal=kudu/127.18.154.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.154.2:44659
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.154.2
--webserver_port=44583
--tserver_master_addrs=127.18.154.62:40369
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.154.2
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:15:45.397135 5463 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:15:45.398959 5463 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:15:45.436995 5485 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:15:45.437784 5486 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:15:45.449105 5488 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:15:45.450112 5463 server_base.cc:1034] running on GCE node
I20260807 18:15:45.450973 5463 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:15:45.452391 5463 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:15:45.453663 5463 hybrid_clock.cc:648] HybridClock initialized: now 1786126545453598 us; error 46 us; skew 500 ppm
I20260807 18:15:45.454093 5463 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:15:45 dist-test-slave-gkw7 krb5kdc[3738](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126545, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:15:45.458701 5463 init.cc:377] Logged in from keytab as kudu/127.18.154.2@KRBTEST.COM (short username kudu)
I20260807 18:15:45.460958 5463 webserver.cc:460] Webserver started at http://127.18.154.2:44583/ using document root <none> and password file <none>
I20260807 18:15:45.462044 5463 fs_manager.cc:359] Metadata directory not provided
I20260807 18:15:45.462232 5463 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 07 18:15:45 dist-test-slave-gkw7 krb5kdc[3738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126545, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.2@KRBTEST.COM for HTTP/127.18.154.20@KRBTEST.COM
I20260807 18:15:45.488894 5463 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.004s sys 0.000s
I20260807 18:15:45.492766 5502 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:15:45.494424 5463 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 18:15:45.494637 5463 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/minicluster-data/ts-1/wal
uuid: "ae903e78a8a44c1c9fd3629bcb34d58a"
format_stamp: "Formatted at 2026-08-07 18:15:42 on dist-test-slave-gkw7"
server_key: "af0b47888b5df4ace1ff43766c77cccc"
server_key_iv: "355f6c22f1121e7f5f35bbaeac661692"
server_key_version: "kuduclusterkey@0"
I20260807 18:15:45.495774 5463 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
Aug 07 18:15:45 dist-test-slave-gkw7 krb5kdc[3738](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786126545, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:15:45.518244 5463 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:15:45.523803 5463 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:15:45.524211 5463 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:15:45.528992 5463 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:15:45.531025 5463 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:15:45.531227 5463 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20260807 18:15:45.531499 5463 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:15:45.531661 5463 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:15:45.586314 5463 rpc_server.cc:307] RPC server started. Bound to: 127.18.154.2:44659
I20260807 18:15:45.586385 5658 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.154.2:44659 every 8 connection(s)
I20260807 18:15:45.588327 5463 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/minicluster-data/ts-1/data/info.pb
I20260807 18:15:45.592674 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 5463
I20260807 18:15:45.593803 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.154.20:36567/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.154.3:44187
--local_ip_for_outbound_sockets=127.18.154.3
--tserver_master_addrs=127.18.154.62:40369
--webserver_port=40275
--webserver_interface=127.18.154.3
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.3.keytab
--principal=kudu/127.18.154.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.154.20:46707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.18.154.20:36567/kms with env {KRB5CCNAME=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 18:15:45 dist-test-slave-gkw7 krb5kdc[3738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126545, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.2@KRBTEST.COM for kudu/127.18.154.62@KRBTEST.COM
I20260807 18:15:45.623123 5663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:45.592365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:40369 (local address 127.18.154.2:57655)
0807 18:15:45.598556 (+ 6191us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:45.598616 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0807 18:15:45.600471 (+ 1855us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:15:45.601814 (+ 1343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:15:45.601832 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:15:45.602505 (+ 673us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:15:45.603119 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:45.603138 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:45.604284 (+ 1146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:45.604292 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:45.605603 (+ 1311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:45.605633 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:45.606851 (+ 1218us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:45.606859 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:45.607199 (+ 340us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:45.608827 (+ 1628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:15:45.608863 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:15:45.612006 (+ 3143us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:15:45.616606 (+ 4600us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:15:45.616618 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:15:45.616639 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:15:45.617337 (+ 698us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:15:45.617900 (+ 563us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:15:45.617905 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:15:45.617909 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:15:45.618174 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:15:45.618929 (+ 755us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:15:45.618939 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:15:45.620387 (+ 1448us) client_negotiation.cc:769] Sending connection context
0807 18:15:45.620790 (+ 403us) client_negotiation.cc:240] Negotiation successful
0807 18:15:45.621464 (+ 674us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5871,"thread_start_us":5650,"threads_started":1}
I20260807 18:15:45.624529 5662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:45.591786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.2:57655 (local address 127.18.154.62:40369)
0807 18:15:45.601098 (+ 9312us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:45.601105 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:45.601126 (+ 21us) server_negotiation.cc:407] Connection header received
0807 18:15:45.601198 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:45.601204 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:45.601282 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:45.601411 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:15:45.603323 (+ 1912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:45.604113 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:45.605771 (+ 1658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:45.606321 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:45.606481 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:45.612322 (+ 5841us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:15:45.612329 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:15:45.612334 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:15:45.612398 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:15:45.616400 (+ 4002us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:15:45.617565 (+ 1165us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:15:45.617572 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:15:45.617577 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:15:45.617743 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:15:45.618340 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:15:45.618346 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:15:45.618350 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:15:45.618754 (+ 404us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:15:45.623686 (+ 4932us) server_negotiation.cc:1035] Waiting for connection context
0807 18:15:45.623841 (+ 155us) server_negotiation.cc:299] Negotiation successful
0807 18:15:45.624107 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9152,"thread_start_us":9042,"threads_started":1}
I20260807 18:15:45.626335 5659 heartbeater.cc:347] Connected to a master server at 127.18.154.62:40369
I20260807 18:15:45.626724 5659 heartbeater.cc:464] Registering TS with master...
I20260807 18:15:45.627729 5659 heartbeater.cc:511] Master 127.18.154.62:40369 requested a full tablet report, sending...
I20260807 18:15:45.631043 5174 ts_manager.cc:194] Registered new tserver with Master: ae903e78a8a44c1c9fd3629bcb34d58a (127.18.154.2:44659)
I20260807 18:15:45.633517 5174 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.154.2@KRBTEST.COM'} at 127.18.154.2:57655
I20260807 18:15:45.947156 5457 heartbeater.cc:464] Registering TS with master...
I20260807 18:15:45.947669 5457 heartbeater.cc:511] Master 127.18.154.62:40369 requested a full tablet report, sending...
I20260807 18:15:45.949079 5174 ts_manager.cc:194] Registered new tserver with Master: 37214b71f71a4a17901e6cbafd0a44d2 (127.18.154.1:34641)
I20260807 18:15:45.950589 5174 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.154.1@KRBTEST.COM'} at 127.18.154.1:37157
Aug 07 18:15:46 dist-test-slave-gkw7 krb5kdc[3738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126545, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.62@KRBTEST.COM for HTTP/127.18.154.20@KRBTEST.COM
W20260807 18:15:46.106169 5666 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:15:46.106608 5666 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:15:46.106823 5666 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:15:46.115298 5666 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:15:46.115465 5666 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:15:46.115660 5666 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.154.3
I20260807 18:15:46.126643 5666 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.154.20:46707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.18.154.20:36567/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.3.keytab
--principal=kudu/127.18.154.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.154.3:44187
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.154.3
--webserver_port=40275
--tserver_master_addrs=127.18.154.62:40369
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.154.3
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:15:46.128088 5666 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:15:46.129684 5666 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:15:46.145377 5671 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:15:46.146467 5673 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:15:46.153818 5666 server_base.cc:1034] running on GCE node
W20260807 18:15:46.154862 5683 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:15:46.155750 5666 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:15:46.157198 5666 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:15:46.158526 5666 hybrid_clock.cc:648] HybridClock initialized: now 1786126546158448 us; error 58 us; skew 500 ppm
I20260807 18:15:46.158974 5666 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:15:46 dist-test-slave-gkw7 krb5kdc[3738](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126546, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:15:46.163645 5666 init.cc:377] Logged in from keytab as kudu/127.18.154.3@KRBTEST.COM (short username kudu)
I20260807 18:15:46.167075 5666 webserver.cc:460] Webserver started at http://127.18.154.3:40275/ using document root <none> and password file <none>
I20260807 18:15:46.168030 5666 fs_manager.cc:359] Metadata directory not provided
I20260807 18:15:46.168181 5666 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 07 18:15:46 dist-test-slave-gkw7 krb5kdc[3738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126546, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.3@KRBTEST.COM for HTTP/127.18.154.20@KRBTEST.COM
I20260807 18:15:46.195565 5666 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.000s sys 0.004s
I20260807 18:15:46.199523 5693 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:15:46.201637 5666 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.004s
I20260807 18:15:46.201853 5666 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/minicluster-data/ts-2/wal
uuid: "42ed1c0b90f544d29feabe07cea7528d"
format_stamp: "Formatted at 2026-08-07 18:15:42 on dist-test-slave-gkw7"
server_key: "e274a96d2c9cb23befee18c1929ba6de"
server_key_iv: "f4e66858071297c7efb7b79bbf2e4ea3"
server_key_version: "kuduclusterkey@0"
I20260807 18:15:46.203341 5666 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:15:46.226032 5666 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:15:46.231360 5666 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:15:46.231667 5666 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:15:46.233135 5666 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:15:46.235077 5666 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:15:46.235224 5666 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:15:46.235368 5666 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:15:46.235455 5666 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:15:46.297122 5666 rpc_server.cc:307] RPC server started. Bound to: 127.18.154.3:44187
I20260807 18:15:46.297179 5843 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.154.3:44187 every 8 connection(s)
I20260807 18:15:46.299294 5666 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/minicluster-data/ts-2/data/info.pb
I20260807 18:15:46.304261 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 5666
Aug 07 18:15:46 dist-test-slave-gkw7 krb5kdc[3738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126546, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.3@KRBTEST.COM for kudu/127.18.154.62@KRBTEST.COM
I20260807 18:15:46.370463 5849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:46.305356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52998 (local address 127.18.154.62:40369)
0807 18:15:46.345209 (+ 39853us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:46.345217 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:46.345245 (+ 28us) server_negotiation.cc:407] Connection header received
0807 18:15:46.345352 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:46.345362 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:46.345497 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:46.345716 (+ 219us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:15:46.349733 (+ 4017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:46.351335 (+ 1602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:46.353967 (+ 2632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:46.354940 (+ 973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:46.355137 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:46.358903 (+ 3766us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:15:46.358910 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:15:46.358915 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:15:46.358963 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:15:46.365860 (+ 6897us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:15:46.366785 (+ 925us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:15:46.366792 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:15:46.366796 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:15:46.366960 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:15:46.367390 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:15:46.367396 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:15:46.367400 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:15:46.367764 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:15:46.368035 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0807 18:15:46.369784 (+ 1749us) server_negotiation.cc:299] Negotiation successful
0807 18:15:46.370075 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39684,"thread_start_us":38669,"threads_started":1}
I20260807 18:15:46.381249 5850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:46.305030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.3:48347 (local address 127.18.154.62:40369)
0807 18:15:46.343974 (+ 38944us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:46.343984 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:46.349624 (+ 5640us) server_negotiation.cc:407] Connection header received
0807 18:15:46.356535 (+ 6911us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:46.356540 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:46.356662 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:46.356771 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:15:46.359355 (+ 2584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:46.360596 (+ 1241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:46.363073 (+ 2477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:46.363792 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:46.363904 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:46.371142 (+ 7238us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:15:46.371149 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:15:46.371154 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:15:46.371196 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:15:46.377237 (+ 6041us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:15:46.378365 (+ 1128us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:15:46.378370 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:15:46.378374 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:15:46.378522 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:15:46.379102 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:15:46.379107 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:15:46.379110 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:15:46.379464 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:15:46.379593 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0807 18:15:46.380639 (+ 1046us) server_negotiation.cc:299] Negotiation successful
0807 18:15:46.380894 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38740,"thread_start_us":40897,"threads_started":1}
I20260807 18:15:46.384104 5847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:46.303293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:40369 (local address 127.18.154.3:48347)
0807 18:15:46.346891 (+ 43598us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:46.346986 (+ 95us) client_negotiation.cc:174] Beginning negotiation
0807 18:15:46.356227 (+ 9241us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:15:46.357344 (+ 1117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:15:46.357359 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:15:46.358111 (+ 752us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:15:46.359069 (+ 958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:46.359098 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:46.360799 (+ 1701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:46.360804 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:46.362865 (+ 2061us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:46.362890 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:46.364298 (+ 1408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:46.364305 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:46.364611 (+ 306us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:46.365373 (+ 762us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:15:46.365415 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:15:46.370834 (+ 5419us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:15:46.377437 (+ 6603us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:15:46.377448 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:15:46.377469 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:15:46.378211 (+ 742us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:15:46.378688 (+ 477us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:15:46.378692 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:15:46.378695 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:15:46.378968 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:15:46.379711 (+ 743us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:15:46.379719 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:15:46.380189 (+ 470us) client_negotiation.cc:769] Sending connection context
0807 18:15:46.381949 (+ 1760us) client_negotiation.cc:240] Negotiation successful
0807 18:15:46.382573 (+ 624us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43107,"thread_start_us":48125,"threads_started":1}
I20260807 18:15:46.386713 5844 heartbeater.cc:347] Connected to a master server at 127.18.154.62:40369
I20260807 18:15:46.387074 5844 heartbeater.cc:464] Registering TS with master...
I20260807 18:15:46.388044 5844 heartbeater.cc:511] Master 127.18.154.62:40369 requested a full tablet report, sending...
I20260807 18:15:46.389940 5174 ts_manager.cc:194] Registered new tserver with Master: 42ed1c0b90f544d29feabe07cea7528d (127.18.154.3:44187)
I20260807 18:15:46.391031 5174 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.154.3@KRBTEST.COM'} at 127.18.154.3:48347
I20260807 18:15:46.392498 19048 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 18:15:46 dist-test-slave-gkw7 krb5kdc[3738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126545, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.62@KRBTEST.COM for HTTP/127.18.154.20@KRBTEST.COM
I20260807 18:15:46.430305 5850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:46.397219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53006 (local address 127.18.154.62:40369)
0807 18:15:46.397389 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:46.397395 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:46.410131 (+ 12736us) server_negotiation.cc:407] Connection header received
0807 18:15:46.410384 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:46.410391 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:46.410486 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:46.410591 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:15:46.416398 (+ 5807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:46.417508 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:46.419545 (+ 2037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:46.420229 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:46.420341 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:46.423944 (+ 3603us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:15:46.423950 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:15:46.423953 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:15:46.423995 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:15:46.426856 (+ 2861us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:15:46.427837 (+ 981us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:15:46.427843 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:15:46.427846 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:15:46.428015 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:15:46.428514 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:15:46.428519 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:15:46.428523 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:15:46.428994 (+ 471us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:15:46.429275 (+ 281us) server_negotiation.cc:1035] Waiting for connection context
0807 18:15:46.429661 (+ 386us) server_negotiation.cc:299] Negotiation successful
0807 18:15:46.429916 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260807 18:15:46.436712 5174 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53006:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 18:15:46.446286 5174 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:15:46.526742 5868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:46.488760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.1:34641 (local address 127.0.0.1:53234)
0807 18:15:46.514121 (+ 25361us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:46.514150 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0807 18:15:46.514337 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:15:46.515674 (+ 1337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:15:46.515682 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:15:46.515720 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:15:46.516245 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:46.516259 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:46.520071 (+ 3812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:46.520078 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:46.522660 (+ 2582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:46.522675 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:46.524973 (+ 2298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:46.524980 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:46.525951 (+ 971us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:46.526009 (+ 58us) client_negotiation.cc:769] Sending connection context
0807 18:15:46.526147 (+ 138us) client_negotiation.cc:240] Negotiation successful
0807 18:15:46.526338 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25172,"mutex_wait_us":758,"thread_start_us":14177,"threads_started":1}
I20260807 18:15:46.527658 5870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:46.489123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53234 (local address 127.18.154.1:34641)
0807 18:15:46.502728 (+ 13605us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:46.502739 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:46.514521 (+ 11782us) server_negotiation.cc:407] Connection header received
0807 18:15:46.514690 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:46.514704 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:46.515080 (+ 376us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:46.515353 (+ 273us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:15:46.516492 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:46.518693 (+ 2201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:46.522857 (+ 4164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:46.524409 (+ 1552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:46.524554 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:46.524804 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0807 18:15:46.527166 (+ 2362us) server_negotiation.cc:299] Negotiation successful
0807 18:15:46.527349 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13378,"thread_start_us":14329,"threads_started":1}
I20260807 18:15:46.532608 5361 tablet_service.cc:1467] Processing CreateTablet for tablet b29f3732493648b4ba615d648a628c4d (DEFAULT_TABLE table=test-table [id=82a024c2a43149dbb46e47757da159a5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:15:46.534346 5361 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b29f3732493648b4ba615d648a628c4d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:15:46.550112 5869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:46.488247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.2:44659 (local address 127.0.0.1:50470)
0807 18:15:46.503415 (+ 15168us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:46.503478 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0807 18:15:46.503690 (+ 212us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:15:46.519815 (+ 16125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:15:46.519820 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:15:46.519846 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:15:46.520306 (+ 460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:46.520325 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:46.522337 (+ 2012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:46.522343 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:46.525711 (+ 3368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:46.525726 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:46.549203 (+ 23477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:46.549214 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:46.549416 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:46.549457 (+ 41us) client_negotiation.cc:769] Sending connection context
0807 18:15:46.549571 (+ 114us) client_negotiation.cc:240] Negotiation successful
0807 18:15:46.549743 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14823,"thread_start_us":25500,"threads_started":1}
I20260807 18:15:46.551394 5872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:46.489136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.3:44187 (local address 127.0.0.1:57698)
0807 18:15:46.533374 (+ 44238us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:46.533397 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 18:15:46.533573 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:15:46.534630 (+ 1057us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:15:46.534636 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:15:46.534661 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:15:46.534968 (+ 307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:46.534979 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:46.538830 (+ 3851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:46.538836 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:46.540951 (+ 2115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:46.540969 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:46.550554 (+ 9585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:46.550561 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:46.550763 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:46.550799 (+ 36us) client_negotiation.cc:769] Sending connection context
0807 18:15:46.550892 (+ 93us) client_negotiation.cc:240] Negotiation successful
0807 18:15:46.551034 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42802,"thread_start_us":55474,"threads_started":1}
I20260807 18:15:46.551589 5867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:46.488299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50470 (local address 127.18.154.2:44659)
0807 18:15:46.519278 (+ 30979us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:46.519292 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:46.519316 (+ 24us) server_negotiation.cc:407] Connection header received
0807 18:15:46.519393 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:46.519398 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:46.519595 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:46.519839 (+ 244us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:15:46.520604 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:46.522183 (+ 1579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:46.545734 (+ 23551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:46.547239 (+ 1505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:46.550799 (+ 3560us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:46.551024 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0807 18:15:46.551105 (+ 81us) server_negotiation.cc:299] Negotiation successful
0807 18:15:46.551287 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30809,"thread_start_us":30704,"threads_started":1}
I20260807 18:15:46.552574 5876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:46.489797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57698 (local address 127.18.154.3:44187)
0807 18:15:46.531463 (+ 41666us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:46.531472 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:46.533708 (+ 2236us) server_negotiation.cc:407] Connection header received
0807 18:15:46.533814 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:46.533821 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:46.534058 (+ 237us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:46.534323 (+ 265us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:15:46.536194 (+ 1871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:46.538648 (+ 2454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:46.544912 (+ 6264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:46.549938 (+ 5026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:46.550099 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:46.550370 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0807 18:15:46.552034 (+ 1664us) server_negotiation.cc:299] Negotiation successful
0807 18:15:46.552232 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41473,"thread_start_us":41692,"threads_started":1}
I20260807 18:15:46.556190 5574 tablet_service.cc:1467] Processing CreateTablet for tablet b29f3732493648b4ba615d648a628c4d (DEFAULT_TABLE table=test-table [id=82a024c2a43149dbb46e47757da159a5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:15:46.556949 5755 tablet_service.cc:1467] Processing CreateTablet for tablet b29f3732493648b4ba615d648a628c4d (DEFAULT_TABLE table=test-table [id=82a024c2a43149dbb46e47757da159a5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:15:46.557777 5574 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b29f3732493648b4ba615d648a628c4d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:15:46.558696 5755 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b29f3732493648b4ba615d648a628c4d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:15:46.567889 5879 tablet_bootstrap.cc:492] T b29f3732493648b4ba615d648a628c4d P 37214b71f71a4a17901e6cbafd0a44d2: Bootstrap starting.
I20260807 18:15:46.574054 5879 tablet_bootstrap.cc:654] T b29f3732493648b4ba615d648a628c4d P 37214b71f71a4a17901e6cbafd0a44d2: Neither blocks nor log segments found. Creating new log.
I20260807 18:15:46.575754 5879 log.cc:824] T b29f3732493648b4ba615d648a628c4d P 37214b71f71a4a17901e6cbafd0a44d2: Log is configured to *not* fsync() on all Append() calls
I20260807 18:15:46.587002 5879 tablet_bootstrap.cc:492] T b29f3732493648b4ba615d648a628c4d P 37214b71f71a4a17901e6cbafd0a44d2: No bootstrap required, opened a new log
I20260807 18:15:46.587437 5879 ts_tablet_manager.cc:1397] T b29f3732493648b4ba615d648a628c4d P 37214b71f71a4a17901e6cbafd0a44d2: Time spent bootstrapping tablet: real 0.020s user 0.001s sys 0.014s
I20260807 18:15:46.596186 5882 tablet_bootstrap.cc:492] T b29f3732493648b4ba615d648a628c4d P 42ed1c0b90f544d29feabe07cea7528d: Bootstrap starting.
I20260807 18:15:46.598482 5879 raft_consensus.cc:348] T b29f3732493648b4ba615d648a628c4d P 37214b71f71a4a17901e6cbafd0a44d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae903e78a8a44c1c9fd3629bcb34d58a" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 44659 } } peers { permanent_uuid: "37214b71f71a4a17901e6cbafd0a44d2" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 34641 } } peers { permanent_uuid: "42ed1c0b90f544d29feabe07cea7528d" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 44187 } }
I20260807 18:15:46.599134 5879 raft_consensus.cc:374] T b29f3732493648b4ba615d648a628c4d P 37214b71f71a4a17901e6cbafd0a44d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:15:46.599308 5879 raft_consensus.cc:729] T b29f3732493648b4ba615d648a628c4d P 37214b71f71a4a17901e6cbafd0a44d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 37214b71f71a4a17901e6cbafd0a44d2, State: Initialized, Role: FOLLOWER
I20260807 18:15:46.600087 5879 consensus_queue.cc:260] T b29f3732493648b4ba615d648a628c4d P 37214b71f71a4a17901e6cbafd0a44d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae903e78a8a44c1c9fd3629bcb34d58a" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 44659 } } peers { permanent_uuid: "37214b71f71a4a17901e6cbafd0a44d2" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 34641 } } peers { permanent_uuid: "42ed1c0b90f544d29feabe07cea7528d" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 44187 } }
I20260807 18:15:46.617142 5882 tablet_bootstrap.cc:654] T b29f3732493648b4ba615d648a628c4d P 42ed1c0b90f544d29feabe07cea7528d: Neither blocks nor log segments found. Creating new log.
I20260807 18:15:46.618800 5885 tablet_bootstrap.cc:492] T b29f3732493648b4ba615d648a628c4d P ae903e78a8a44c1c9fd3629bcb34d58a: Bootstrap starting.
I20260807 18:15:46.618942 5882 log.cc:824] T b29f3732493648b4ba615d648a628c4d P 42ed1c0b90f544d29feabe07cea7528d: Log is configured to *not* fsync() on all Append() calls
I20260807 18:15:46.619999 5457 heartbeater.cc:503] Master 127.18.154.62:40369 was elected leader, sending a full tablet report...
I20260807 18:15:46.621122 5879 ts_tablet_manager.cc:1428] T b29f3732493648b4ba615d648a628c4d P 37214b71f71a4a17901e6cbafd0a44d2: Time spent starting tablet: real 0.033s user 0.020s sys 0.010s
I20260807 18:15:46.624388 5885 tablet_bootstrap.cc:654] T b29f3732493648b4ba615d648a628c4d P ae903e78a8a44c1c9fd3629bcb34d58a: Neither blocks nor log segments found. Creating new log.
I20260807 18:15:46.625892 5885 log.cc:824] T b29f3732493648b4ba615d648a628c4d P ae903e78a8a44c1c9fd3629bcb34d58a: Log is configured to *not* fsync() on all Append() calls
I20260807 18:15:46.635824 5885 tablet_bootstrap.cc:492] T b29f3732493648b4ba615d648a628c4d P ae903e78a8a44c1c9fd3629bcb34d58a: No bootstrap required, opened a new log
I20260807 18:15:46.636256 5885 ts_tablet_manager.cc:1397] T b29f3732493648b4ba615d648a628c4d P ae903e78a8a44c1c9fd3629bcb34d58a: Time spent bootstrapping tablet: real 0.018s user 0.015s sys 0.000s
I20260807 18:15:46.638736 5659 heartbeater.cc:503] Master 127.18.154.62:40369 was elected leader, sending a full tablet report...
I20260807 18:15:46.643499 5882 tablet_bootstrap.cc:492] T b29f3732493648b4ba615d648a628c4d P 42ed1c0b90f544d29feabe07cea7528d: No bootstrap required, opened a new log
I20260807 18:15:46.643842 5882 ts_tablet_manager.cc:1397] T b29f3732493648b4ba615d648a628c4d P 42ed1c0b90f544d29feabe07cea7528d: Time spent bootstrapping tablet: real 0.048s user 0.008s sys 0.007s
I20260807 18:15:46.645880 5885 raft_consensus.cc:348] T b29f3732493648b4ba615d648a628c4d P ae903e78a8a44c1c9fd3629bcb34d58a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae903e78a8a44c1c9fd3629bcb34d58a" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 44659 } } peers { permanent_uuid: "37214b71f71a4a17901e6cbafd0a44d2" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 34641 } } peers { permanent_uuid: "42ed1c0b90f544d29feabe07cea7528d" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 44187 } }
I20260807 18:15:46.646449 5885 raft_consensus.cc:374] T b29f3732493648b4ba615d648a628c4d P ae903e78a8a44c1c9fd3629bcb34d58a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:15:46.646634 5885 raft_consensus.cc:729] T b29f3732493648b4ba615d648a628c4d P ae903e78a8a44c1c9fd3629bcb34d58a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae903e78a8a44c1c9fd3629bcb34d58a, State: Initialized, Role: FOLLOWER
I20260807 18:15:46.647498 5885 consensus_queue.cc:260] T b29f3732493648b4ba615d648a628c4d P ae903e78a8a44c1c9fd3629bcb34d58a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae903e78a8a44c1c9fd3629bcb34d58a" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 44659 } } peers { permanent_uuid: "37214b71f71a4a17901e6cbafd0a44d2" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 34641 } } peers { permanent_uuid: "42ed1c0b90f544d29feabe07cea7528d" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 44187 } }
I20260807 18:15:46.656211 5885 ts_tablet_manager.cc:1428] T b29f3732493648b4ba615d648a628c4d P ae903e78a8a44c1c9fd3629bcb34d58a: Time spent starting tablet: real 0.020s user 0.013s sys 0.004s
W20260807 18:15:46.656924 5458 tablet.cc:2369] T b29f3732493648b4ba615d648a628c4d P 37214b71f71a4a17901e6cbafd0a44d2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:15:46.662300 5882 raft_consensus.cc:348] T b29f3732493648b4ba615d648a628c4d P 42ed1c0b90f544d29feabe07cea7528d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae903e78a8a44c1c9fd3629bcb34d58a" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 44659 } } peers { permanent_uuid: "37214b71f71a4a17901e6cbafd0a44d2" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 34641 } } peers { permanent_uuid: "42ed1c0b90f544d29feabe07cea7528d" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 44187 } }
I20260807 18:15:46.662842 5882 raft_consensus.cc:374] T b29f3732493648b4ba615d648a628c4d P 42ed1c0b90f544d29feabe07cea7528d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:15:46.662959 5882 raft_consensus.cc:729] T b29f3732493648b4ba615d648a628c4d P 42ed1c0b90f544d29feabe07cea7528d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42ed1c0b90f544d29feabe07cea7528d, State: Initialized, Role: FOLLOWER
I20260807 18:15:46.663707 5882 consensus_queue.cc:260] T b29f3732493648b4ba615d648a628c4d P 42ed1c0b90f544d29feabe07cea7528d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae903e78a8a44c1c9fd3629bcb34d58a" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 44659 } } peers { permanent_uuid: "37214b71f71a4a17901e6cbafd0a44d2" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 34641 } } peers { permanent_uuid: "42ed1c0b90f544d29feabe07cea7528d" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 44187 } }
I20260807 18:15:46.664973 5844 heartbeater.cc:503] Master 127.18.154.62:40369 was elected leader, sending a full tablet report...
I20260807 18:15:46.665810 5882 ts_tablet_manager.cc:1428] T b29f3732493648b4ba615d648a628c4d P 42ed1c0b90f544d29feabe07cea7528d: Time spent starting tablet: real 0.022s user 0.017s sys 0.000s
I20260807 18:15:46.671039 5891 raft_consensus.cc:482] T b29f3732493648b4ba615d648a628c4d P 42ed1c0b90f544d29feabe07cea7528d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:15:46.671324 5891 raft_consensus.cc:504] T b29f3732493648b4ba615d648a628c4d P 42ed1c0b90f544d29feabe07cea7528d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae903e78a8a44c1c9fd3629bcb34d58a" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 44659 } } peers { permanent_uuid: "37214b71f71a4a17901e6cbafd0a44d2" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 34641 } } peers { permanent_uuid: "42ed1c0b90f544d29feabe07cea7528d" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 44187 } }
I20260807 18:15:46.673385 5891 leader_election.cc:288] T b29f3732493648b4ba615d648a628c4d P 42ed1c0b90f544d29feabe07cea7528d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ae903e78a8a44c1c9fd3629bcb34d58a (127.18.154.2:44659), 37214b71f71a4a17901e6cbafd0a44d2 (127.18.154.1:34641)
I20260807 18:15:46.681488 5847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:46.673652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.2:44659 (local address 127.18.154.3:50795)
0807 18:15:46.674756 (+ 1104us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:46.674778 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 18:15:46.674903 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:15:46.675324 (+ 421us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:15:46.675328 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:15:46.675357 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:15:46.675541 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:46.675551 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:46.677049 (+ 1498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:46.677055 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:46.679179 (+ 2124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:46.679196 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:46.680726 (+ 1530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:46.680732 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:46.680940 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:46.680976 (+ 36us) client_negotiation.cc:769] Sending connection context
0807 18:15:46.681072 (+ 96us) client_negotiation.cc:240] Negotiation successful
0807 18:15:46.681206 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":973}
I20260807 18:15:46.682487 5867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:46.674770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.3:50795 (local address 127.18.154.2:44659)
0807 18:15:46.674960 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:46.674966 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:46.674984 (+ 18us) server_negotiation.cc:407] Connection header received
0807 18:15:46.675043 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:46.675046 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:46.675174 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:46.675284 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:15:46.675724 (+ 440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:46.676873 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:46.679348 (+ 2475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:46.680541 (+ 1193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:46.680719 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:46.680795 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0807 18:15:46.682083 (+ 1288us) server_negotiation.cc:299] Negotiation successful
0807 18:15:46.682226 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260807 18:15:46.684149 5599 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b29f3732493648b4ba615d648a628c4d" candidate_uuid: "42ed1c0b90f544d29feabe07cea7528d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ae903e78a8a44c1c9fd3629bcb34d58a" is_pre_election: true
I20260807 18:15:46.684722 5599 raft_consensus.cc:2454] T b29f3732493648b4ba615d648a628c4d P ae903e78a8a44c1c9fd3629bcb34d58a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 42ed1c0b90f544d29feabe07cea7528d in term 0.
I20260807 18:15:46.685491 5696 leader_election.cc:302] T b29f3732493648b4ba615d648a628c4d P 42ed1c0b90f544d29feabe07cea7528d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 42ed1c0b90f544d29feabe07cea7528d, ae903e78a8a44c1c9fd3629bcb34d58a; no voters:
I20260807 18:15:46.686158 5891 raft_consensus.cc:2781] T b29f3732493648b4ba615d648a628c4d P 42ed1c0b90f544d29feabe07cea7528d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:15:46.686390 5891 raft_consensus.cc:482] T b29f3732493648b4ba615d648a628c4d P 42ed1c0b90f544d29feabe07cea7528d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:15:46.686548 5891 raft_consensus.cc:3037] T b29f3732493648b4ba615d648a628c4d P 42ed1c0b90f544d29feabe07cea7528d [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:15:46.689924 5870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:46.675381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.3:33323 (local address 127.18.154.1:34641)
0807 18:15:46.675932 (+ 551us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:46.675941 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:46.681875 (+ 5934us) server_negotiation.cc:407] Connection header received
0807 18:15:46.682058 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:46.682064 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:46.682169 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:46.682258 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:15:46.682836 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:46.683623 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:46.687414 (+ 3791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:46.688637 (+ 1223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:46.688782 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:46.688857 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0807 18:15:46.689499 (+ 642us) server_negotiation.cc:299] Negotiation successful
0807 18:15:46.689663 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":397}
I20260807 18:15:46.690860 5847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:46.674201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.1:34641 (local address 127.18.154.3:33323)
0807 18:15:46.681755 (+ 7554us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:46.681795 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 18:15:46.681921 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:15:46.682416 (+ 495us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:15:46.682420 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:15:46.682445 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:15:46.682651 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:46.682662 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:46.685236 (+ 2574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:46.685242 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:46.687208 (+ 1966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:46.687224 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:46.689014 (+ 1790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:46.689022 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:46.689301 (+ 279us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:46.689334 (+ 33us) client_negotiation.cc:769] Sending connection context
0807 18:15:46.690335 (+ 1001us) client_negotiation.cc:240] Negotiation successful
0807 18:15:46.690488 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7458,"threads_started":1}
I20260807 18:15:46.690886 5891 raft_consensus.cc:504] T b29f3732493648b4ba615d648a628c4d P 42ed1c0b90f544d29feabe07cea7528d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae903e78a8a44c1c9fd3629bcb34d58a" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 44659 } } peers { permanent_uuid: "37214b71f71a4a17901e6cbafd0a44d2" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 34641 } } peers { permanent_uuid: "42ed1c0b90f544d29feabe07cea7528d" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 44187 } }
I20260807 18:15:46.692210 5599 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b29f3732493648b4ba615d648a628c4d" candidate_uuid: "42ed1c0b90f544d29feabe07cea7528d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ae903e78a8a44c1c9fd3629bcb34d58a"
I20260807 18:15:46.692423 5599 raft_consensus.cc:3037] T b29f3732493648b4ba615d648a628c4d P ae903e78a8a44c1c9fd3629bcb34d58a [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:15:46.692502 5389 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b29f3732493648b4ba615d648a628c4d" candidate_uuid: "42ed1c0b90f544d29feabe07cea7528d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "37214b71f71a4a17901e6cbafd0a44d2" is_pre_election: true
I20260807 18:15:46.693156 5389 raft_consensus.cc:2454] T b29f3732493648b4ba615d648a628c4d P 37214b71f71a4a17901e6cbafd0a44d2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 42ed1c0b90f544d29feabe07cea7528d in term 0.
I20260807 18:15:46.695729 5599 raft_consensus.cc:2454] T b29f3732493648b4ba615d648a628c4d P ae903e78a8a44c1c9fd3629bcb34d58a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 42ed1c0b90f544d29feabe07cea7528d in term 1.
I20260807 18:15:46.696276 5696 leader_election.cc:302] T b29f3732493648b4ba615d648a628c4d P 42ed1c0b90f544d29feabe07cea7528d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 42ed1c0b90f544d29feabe07cea7528d, ae903e78a8a44c1c9fd3629bcb34d58a; no voters:
I20260807 18:15:46.697160 5891 leader_election.cc:288] T b29f3732493648b4ba615d648a628c4d P 42ed1c0b90f544d29feabe07cea7528d [CANDIDATE]: Term 1 election: Requested vote from peers ae903e78a8a44c1c9fd3629bcb34d58a (127.18.154.2:44659), 37214b71f71a4a17901e6cbafd0a44d2 (127.18.154.1:34641)
I20260807 18:15:46.697419 5891 raft_consensus.cc:2781] T b29f3732493648b4ba615d648a628c4d P 42ed1c0b90f544d29feabe07cea7528d [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:15:46.697636 5891 raft_consensus.cc:686] T b29f3732493648b4ba615d648a628c4d P 42ed1c0b90f544d29feabe07cea7528d [term 1 LEADER]: Becoming Leader. State: Replica: 42ed1c0b90f544d29feabe07cea7528d, State: Running, Role: LEADER
I20260807 18:15:46.698230 5891 consensus_queue.cc:237] T b29f3732493648b4ba615d648a628c4d P 42ed1c0b90f544d29feabe07cea7528d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae903e78a8a44c1c9fd3629bcb34d58a" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 44659 } } peers { permanent_uuid: "37214b71f71a4a17901e6cbafd0a44d2" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 34641 } } peers { permanent_uuid: "42ed1c0b90f544d29feabe07cea7528d" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 44187 } }
I20260807 18:15:46.703073 5389 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b29f3732493648b4ba615d648a628c4d" candidate_uuid: "42ed1c0b90f544d29feabe07cea7528d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "37214b71f71a4a17901e6cbafd0a44d2"
I20260807 18:15:46.703315 5389 raft_consensus.cc:3037] T b29f3732493648b4ba615d648a628c4d P 37214b71f71a4a17901e6cbafd0a44d2 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:15:46.706979 5389 raft_consensus.cc:2454] T b29f3732493648b4ba615d648a628c4d P 37214b71f71a4a17901e6cbafd0a44d2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 42ed1c0b90f544d29feabe07cea7528d in term 1.
I20260807 18:15:46.711303 5174 catalog_manager.cc:5495] T b29f3732493648b4ba615d648a628c4d P 42ed1c0b90f544d29feabe07cea7528d reported cstate change: term changed from 0 to 1, leader changed from <none> to 42ed1c0b90f544d29feabe07cea7528d (127.18.154.3). New cstate: current_term: 1 leader_uuid: "42ed1c0b90f544d29feabe07cea7528d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae903e78a8a44c1c9fd3629bcb34d58a" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 44659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "37214b71f71a4a17901e6cbafd0a44d2" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "42ed1c0b90f544d29feabe07cea7528d" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 44187 } health_report { overall_health: HEALTHY } } }
I20260807 18:15:46.737954 5876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:46.731256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57700 (local address 127.18.154.3:44187)
0807 18:15:46.731904 (+ 648us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:46.731910 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:46.731930 (+ 20us) server_negotiation.cc:407] Connection header received
0807 18:15:46.732038 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:46.732043 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:46.732172 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:46.732337 (+ 165us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:15:46.732855 (+ 518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:46.734014 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:46.735552 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:46.735951 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:46.736090 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:46.736716 (+ 626us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:15:46.737283 (+ 567us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:15:46.737473 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0807 18:15:46.737622 (+ 149us) server_negotiation.cc:299] Negotiation successful
0807 18:15:46.737700 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":486}
I20260807 18:15:46.746574 5599 raft_consensus.cc:1261] T b29f3732493648b4ba615d648a628c4d P ae903e78a8a44c1c9fd3629bcb34d58a [term 1 FOLLOWER]: Refusing update from remote peer 42ed1c0b90f544d29feabe07cea7528d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:15:46.746616 5389 raft_consensus.cc:1261] T b29f3732493648b4ba615d648a628c4d P 37214b71f71a4a17901e6cbafd0a44d2 [term 1 FOLLOWER]: Refusing update from remote peer 42ed1c0b90f544d29feabe07cea7528d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:15:46.747789 5894 consensus_queue.cc:1035] T b29f3732493648b4ba615d648a628c4d P 42ed1c0b90f544d29feabe07cea7528d [LEADER]: Connected to new peer: Peer: permanent_uuid: "ae903e78a8a44c1c9fd3629bcb34d58a" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 44659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:15:46.748719 5894 consensus_queue.cc:1035] T b29f3732493648b4ba615d648a628c4d P 42ed1c0b90f544d29feabe07cea7528d [LEADER]: Connected to new peer: Peer: permanent_uuid: "37214b71f71a4a17901e6cbafd0a44d2" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:15:46.765123 5915 mvcc.cc:205] Tried to move back new op lower bound from 7315974335462920192 to 7315974335280832512. Current Snapshot: MvccSnapshot[applied={T|T < 7315974335462920192}]
I20260807 18:15:46.778229 5173 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53006:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:15:46.778595 5173 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53006:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:15:46.782923 5173 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bf02f6e8e4fb4234877a7e85aa40706f: Sending DeleteTablet for 3 replicas of tablet b29f3732493648b4ba615d648a628c4d
I20260807 18:15:46.784310 5361 tablet_service.cc:1514] Processing DeleteTablet for tablet b29f3732493648b4ba615d648a628c4d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:15:46 UTC) from {username='kudu', principal='kudu/127.18.154.62@KRBTEST.COM'} at 127.0.0.1:53234
I20260807 18:15:46.784313 5574 tablet_service.cc:1514] Processing DeleteTablet for tablet b29f3732493648b4ba615d648a628c4d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:15:46 UTC) from {username='kudu', principal='kudu/127.18.154.62@KRBTEST.COM'} at 127.0.0.1:50470
I20260807 18:15:46.785856 5755 tablet_service.cc:1514] Processing DeleteTablet for tablet b29f3732493648b4ba615d648a628c4d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:15:46 UTC) from {username='kudu', principal='kudu/127.18.154.62@KRBTEST.COM'} at 127.0.0.1:57698
I20260807 18:15:46.796301 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 5226
I20260807 18:15:46.798707 5927 tablet_replica.cc:330] T b29f3732493648b4ba615d648a628c4d P 37214b71f71a4a17901e6cbafd0a44d2: stopping tablet replica
I20260807 18:15:46.799396 5927 raft_consensus.cc:2229] T b29f3732493648b4ba615d648a628c4d P 37214b71f71a4a17901e6cbafd0a44d2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:15:46.800112 5927 raft_consensus.cc:2258] T b29f3732493648b4ba615d648a628c4d P 37214b71f71a4a17901e6cbafd0a44d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:15:46.803360 5927 ts_tablet_manager.cc:1905] T b29f3732493648b4ba615d648a628c4d P 37214b71f71a4a17901e6cbafd0a44d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:15:46.803706 5925 tablet_replica.cc:330] T b29f3732493648b4ba615d648a628c4d P ae903e78a8a44c1c9fd3629bcb34d58a: stopping tablet replica
I20260807 18:15:46.804237 5925 raft_consensus.cc:2229] T b29f3732493648b4ba615d648a628c4d P ae903e78a8a44c1c9fd3629bcb34d58a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:15:46.804956 5928 tablet_replica.cc:330] T b29f3732493648b4ba615d648a628c4d P 42ed1c0b90f544d29feabe07cea7528d: stopping tablet replica
I20260807 18:15:46.805397 5928 raft_consensus.cc:2229] T b29f3732493648b4ba615d648a628c4d P 42ed1c0b90f544d29feabe07cea7528d [term 1 LEADER]: Raft consensus shutting down.
I20260807 18:15:46.806313 5928 raft_consensus.cc:2258] T b29f3732493648b4ba615d648a628c4d P 42ed1c0b90f544d29feabe07cea7528d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:15:46.806567 5925 raft_consensus.cc:2258] T b29f3732493648b4ba615d648a628c4d P ae903e78a8a44c1c9fd3629bcb34d58a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:15:46.809964 5925 ts_tablet_manager.cc:1905] T b29f3732493648b4ba615d648a628c4d P ae903e78a8a44c1c9fd3629bcb34d58a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:15:46.814116 5927 ts_tablet_manager.cc:1918] T b29f3732493648b4ba615d648a628c4d P 37214b71f71a4a17901e6cbafd0a44d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:15:46.814392 5927 log.cc:1196] T b29f3732493648b4ba615d648a628c4d P 37214b71f71a4a17901e6cbafd0a44d2: Deleting WAL directory at /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/minicluster-data/ts-0/wal/wals/b29f3732493648b4ba615d648a628c4d
I20260807 18:15:46.815171 5927 ts_tablet_manager.cc:1939] T b29f3732493648b4ba615d648a628c4d P 37214b71f71a4a17901e6cbafd0a44d2: Deleting consensus metadata
I20260807 18:15:46.817584 5158 catalog_manager.cc:4841] TS 37214b71f71a4a17901e6cbafd0a44d2 (127.18.154.1:34641): tablet b29f3732493648b4ba615d648a628c4d (table test-table [id=82a024c2a43149dbb46e47757da159a5]) successfully deleted
I20260807 18:15:46.818724 5925 ts_tablet_manager.cc:1918] T b29f3732493648b4ba615d648a628c4d P ae903e78a8a44c1c9fd3629bcb34d58a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:15:46.818938 5925 log.cc:1196] T b29f3732493648b4ba615d648a628c4d P ae903e78a8a44c1c9fd3629bcb34d58a: Deleting WAL directory at /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/minicluster-data/ts-1/wal/wals/b29f3732493648b4ba615d648a628c4d
I20260807 18:15:46.819285 5928 ts_tablet_manager.cc:1905] T b29f3732493648b4ba615d648a628c4d P 42ed1c0b90f544d29feabe07cea7528d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:15:46.819579 5925 ts_tablet_manager.cc:1939] T b29f3732493648b4ba615d648a628c4d P ae903e78a8a44c1c9fd3629bcb34d58a: Deleting consensus metadata
I20260807 18:15:46.821710 5159 catalog_manager.cc:4841] TS ae903e78a8a44c1c9fd3629bcb34d58a (127.18.154.2:44659): tablet b29f3732493648b4ba615d648a628c4d (table test-table [id=82a024c2a43149dbb46e47757da159a5]) successfully deleted
I20260807 18:15:46.822064 5870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:46.798281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53250 (local address 127.18.154.1:34641)
0807 18:15:46.798568 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:46.798577 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:46.798604 (+ 27us) server_negotiation.cc:407] Connection header received
0807 18:15:46.798703 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:46.798712 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:46.798824 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:46.798963 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:15:46.801834 (+ 2871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:46.803528 (+ 1694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:46.806026 (+ 2498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:46.806778 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:46.806932 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:46.811239 (+ 4307us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:15:46.811255 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:15:46.811275 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:15:46.811332 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:15:46.816183 (+ 4851us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:15:46.818009 (+ 1826us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:15:46.818020 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:15:46.818025 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:15:46.818216 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:15:46.818835 (+ 619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:15:46.818842 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:15:46.818845 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:15:46.819461 (+ 616us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:15:46.819834 (+ 373us) server_negotiation.cc:1035] Waiting for connection context
0807 18:15:46.821269 (+ 1435us) server_negotiation.cc:299] Negotiation successful
0807 18:15:46.821570 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107}
I20260807 18:15:46.822798 5451 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:15:46.827795 5928 ts_tablet_manager.cc:1918] T b29f3732493648b4ba615d648a628c4d P 42ed1c0b90f544d29feabe07cea7528d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:15:46.828013 5928 log.cc:1196] T b29f3732493648b4ba615d648a628c4d P 42ed1c0b90f544d29feabe07cea7528d: Deleting WAL directory at /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786126328808105-19048-0/minicluster-data/ts-2/wal/wals/b29f3732493648b4ba615d648a628c4d
I20260807 18:15:46.828608 5928 ts_tablet_manager.cc:1939] T b29f3732493648b4ba615d648a628c4d P 42ed1c0b90f544d29feabe07cea7528d: Deleting consensus metadata
I20260807 18:15:46.830775 5160 catalog_manager.cc:4841] TS 42ed1c0b90f544d29feabe07cea7528d (127.18.154.3:44187): tablet b29f3732493648b4ba615d648a628c4d (table test-table [id=82a024c2a43149dbb46e47757da159a5]) successfully deleted
I20260807 18:15:46.917874 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 5226
I20260807 18:15:46.940912 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 5463
I20260807 18:15:46.955669 5867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:46.941865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50476 (local address 127.18.154.2:44659)
0807 18:15:46.942077 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:46.942083 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:46.942106 (+ 23us) server_negotiation.cc:407] Connection header received
0807 18:15:46.942190 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:46.942196 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:46.942277 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:46.942395 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:15:46.943906 (+ 1511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:46.944663 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:46.946003 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:46.946416 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:46.946571 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:46.949298 (+ 2727us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:15:46.949314 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:15:46.949332 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:15:46.949388 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:15:46.952363 (+ 2975us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:15:46.953270 (+ 907us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:15:46.953281 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:15:46.953283 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:15:46.953452 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:15:46.953970 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:15:46.953976 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:15:46.953980 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:15:46.954356 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:15:46.954675 (+ 319us) server_negotiation.cc:1035] Waiting for connection context
0807 18:15:46.954989 (+ 314us) server_negotiation.cc:299] Negotiation successful
0807 18:15:46.955216 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260807 18:15:46.956363 5653 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:15:47.058848 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 5463
I20260807 18:15:47.086378 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 5666
I20260807 18:15:47.101884 5876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:47.087236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57706 (local address 127.18.154.3:44187)
0807 18:15:47.087448 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:47.087454 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:47.087586 (+ 132us) server_negotiation.cc:407] Connection header received
0807 18:15:47.087826 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:47.087831 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:47.087906 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:47.088039 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:15:47.089458 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:47.090376 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:47.091735 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:47.092204 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:47.092355 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:47.095222 (+ 2867us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:15:47.095238 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:15:47.095256 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:15:47.095291 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:15:47.098630 (+ 3339us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:15:47.099465 (+ 835us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:15:47.099472 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:15:47.099475 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:15:47.099642 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:15:47.100053 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:15:47.100058 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:15:47.100061 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:15:47.100460 (+ 399us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:15:47.100823 (+ 363us) server_negotiation.cc:1035] Waiting for connection context
0807 18:15:47.101164 (+ 341us) server_negotiation.cc:299] Negotiation successful
0807 18:15:47.101398 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260807 18:15:47.102696 5838 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:15:47.228336 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 5666
I20260807 18:15:47.257951 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 5134
I20260807 18:15:47.259413 5215 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:15:47.378177 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 5134
I20260807 18:15:47.413396 19048 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260807 18:15:47.759625 19048 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260807 18:15:47.759968 19048 mini_ranger.cc:66] Stopping Ranger...
I20260807 18:15:48.106231 19048 mini_ranger.cc:68] Stopped Ranger
2026-08-07 18:15:48.106 UTC [3776] LOG: received smart shutdown request
2026-08-07 18:15:48.110 UTC [3776] LOG: background worker "logical replication launcher" (PID 3790) exited with exit code 1
2026-08-07 18:15:48.110 UTC [3783] LOG: shutting down
2026-08-07 18:15:48.319 UTC [3776] LOG: database system is shut down
2026-08-07T18:15:48Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (56373 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126328808105-19048-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:15:48 dist-test-slave-gkw7 krb5kdc[5945](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:15:48 dist-test-slave-gkw7 krb5kdc[5945](info): set up 2 sockets
Aug 07 18:15:48 dist-test-slave-gkw7 krb5kdc[5945](info): commencing operation
krb5kdc: starting...
W20260807 18:15:49.796087 19048 mini_kdc.cc:121] Time spent starting KDC: real 1.305s user 0.004s sys 0.032s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126328808105-19048-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126328808105-19048-0/krb5kdc/test-user.keytab.
Aug 07 18:15:49 dist-test-slave-gkw7 krb5kdc[5945](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126549, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:15:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:15:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.154.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.154.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.154.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.62.keytab.
Entry for principal kudu/127.18.154.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.62.keytab.
I20260807 18:15:50.089825 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126328808105-19048-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126328808105-19048-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126328808105-19048-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.154.62:34989
--webserver_interface=127.18.154.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.62.keytab
--principal=kudu/127.18.154.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.154.20:32959
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.154.62:34989,127.18.154.61:41087,127.18.154.60:34071
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126328808105-19048-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126328808105-19048-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126328808105-19048-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:15:50.453519 5966 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:15:50.453903 5966 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:15:50.454100 5966 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:15:50.461508 5966 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:15:50.461717 5966 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:15:50.461833 5966 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:15:50.461961 5966 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:15:50.462029 5966 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:15:50.472127 5966 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.154.20:32959
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126328808105-19048-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126328808105-19048-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.154.62:34989,127.18.154.61:41087,127.18.154.60:34071
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.62.keytab
--principal=kudu/127.18.154.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.154.62:34989
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.154.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126328808105-19048-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:15:50.473665 5966 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:15:50.475039 5966 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:15:50.487989 5976 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:15:50.488042 5978 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:15:50.488010 5982 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:15:50.489358 5966 server_base.cc:1034] running on GCE node
I20260807 18:15:50.490326 5966 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:15:50.491884 5966 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:15:50.493121 5966 hybrid_clock.cc:648] HybridClock initialized: now 1786126550493050 us; error 60 us; skew 500 ppm
I20260807 18:15:50.493568 5966 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:15:50 dist-test-slave-gkw7 krb5kdc[5945](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126550, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:15:50.502671 5966 init.cc:377] Logged in from keytab as kudu/127.18.154.62@KRBTEST.COM (short username kudu)
I20260807 18:15:50.504438 5966 webserver.cc:460] Webserver started at http://127.18.154.62:39317/ using document root <none> and password file <none>
I20260807 18:15:50.505285 5966 fs_manager.cc:359] Metadata directory not provided
I20260807 18:15:50.505432 5966 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:15:50.505851 5966 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:15:50.509343 5966 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126328808105-19048-0/minicluster-data/master-0/data/instance:
uuid: "486d3b03a9ee4e8da2514f4845e072a4"
format_stamp: "Formatted at 2026-08-07 18:15:50 on dist-test-slave-gkw7"
server_key: "9921fa434ec5c2bb721ed1685544302c"
server_key_iv: "2b8326ce92042b89296c4f331f036040"
server_key_version: "encryptionkey@0"
I20260807 18:15:50.510344 5966 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126328808105-19048-0/minicluster-data/master-0/wal/instance:
uuid: "486d3b03a9ee4e8da2514f4845e072a4"
format_stamp: "Formatted at 2026-08-07 18:15:50 on dist-test-slave-gkw7"
server_key: "9921fa434ec5c2bb721ed1685544302c"
server_key_iv: "2b8326ce92042b89296c4f331f036040"
server_key_version: "encryptionkey@0"
I20260807 18:15:50.516245 5966 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260807 18:15:50.520334 5990 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:15:50.522261 5966 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260807 18:15:50.522464 5966 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126328808105-19048-0/minicluster-data/master-0/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126328808105-19048-0/minicluster-data/master-0/wal
uuid: "486d3b03a9ee4e8da2514f4845e072a4"
format_stamp: "Formatted at 2026-08-07 18:15:50 on dist-test-slave-gkw7"
server_key: "9921fa434ec5c2bb721ed1685544302c"
server_key_iv: "2b8326ce92042b89296c4f331f036040"
server_key_version: "encryptionkey@0"
I20260807 18:15:50.522686 5966 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126328808105-19048-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126328808105-19048-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126328808105-19048-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:15:50.538604 5966 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:15:50.542999 5966 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:15:50.543305 5966 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:15:50.563251 5966 rpc_server.cc:307] RPC server started. Bound to: 127.18.154.62:34989
I20260807 18:15:50.563301 6064 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.154.62:34989 every 8 connection(s)
I20260807 18:15:50.564746 5966 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
I20260807 18:15:50.568573 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 5966
I20260807 18:15:50.568648 6065 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:15:50.568857 19048 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126328808105-19048-0/minicluster-data/master-0/wal/instance
I20260807 18:15:50.569589 19048 external_mini_cluster.cc:1428] Setting key b30bd06964efe8915834fb427f6e1a06
I20260807 18:15:50.575865 6065 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.154.62" port: 34989 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 18:15:50 dist-test-slave-gkw7 krb5kdc[5945](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126550, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.62@KRBTEST.COM for kudu/127.18.154.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.18.154.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.154.61@KRBTEST.COM" created.
I20260807 18:15:50.600718 6068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:50.579141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:34989 (local address 127.0.0.1:59402)
0807 18:15:50.580098 (+ 957us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:50.580144 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 18:15:50.582188 (+ 2044us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:15:50.583458 (+ 1270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:15:50.583465 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:15:50.584183 (+ 718us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:15:50.584531 (+ 348us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:50.584548 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:50.586202 (+ 1654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:50.586209 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:50.587380 (+ 1171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:50.587393 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:50.588336 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:50.588343 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:50.588581 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:50.589305 (+ 724us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:15:50.589327 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:15:50.591940 (+ 2613us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:15:50.596526 (+ 4586us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:15:50.596538 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:15:50.596553 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:15:50.596993 (+ 440us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:15:50.597485 (+ 492us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:15:50.597488 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:15:50.597491 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:15:50.597714 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:15:50.598505 (+ 791us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:15:50.598511 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:15:50.598741 (+ 230us) client_negotiation.cc:769] Sending connection context
0807 18:15:50.599008 (+ 267us) client_negotiation.cc:240] Negotiation successful
0807 18:15:50.599680 (+ 672us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":686,"mutex_wait_us":249,"thread_start_us":452,"threads_started":1}
I20260807 18:15:50.600785 6069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:50.579141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59402 (local address 127.18.154.62:34989)
0807 18:15:50.582839 (+ 3698us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:50.582844 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:50.582873 (+ 29us) server_negotiation.cc:407] Connection header received
0807 18:15:50.582995 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:50.583000 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:50.583220 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:50.583438 (+ 218us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:15:50.584747 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:50.586039 (+ 1292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:50.587527 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:50.588075 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:50.588287 (+ 212us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:50.592206 (+ 3919us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:15:50.592220 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:15:50.592241 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:15:50.592286 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:15:50.596345 (+ 4059us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:15:50.597124 (+ 779us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:15:50.597131 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:15:50.597133 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:15:50.597378 (+ 245us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:15:50.597815 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:15:50.597818 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:15:50.597821 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:15:50.598358 (+ 537us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:15:50.598555 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0807 18:15:50.599151 (+ 596us) server_negotiation.cc:299] Negotiation successful
0807 18:15:50.599507 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":5,"server-negotiator.queue_time_us":3542,"thread_start_us":3333,"threads_started":1}
I20260807 18:15:50.604259 6065 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.154.61" port: 41087 } has no permanent_uuid. Determining permanent_uuid...
I20260807 18:15:50.605634 6068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:50.605095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.61:41087 (local address 127.0.0.1:44658)
0807 18:15:50.605302 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:50.605427 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.61:41087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
W20260807 18:15:50.605954 5996 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.154.61:41087: connect: Connection refused (error 111)
I20260807 18:15:50.607755 6068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:50.607341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.61:41087 (local address 127.0.0.1:44670)
0807 18:15:50.607540 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:50.607619 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.61:41087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260807 18:15:50.607982 6065 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.154.61:41087: Network error: Client connection negotiation failed: client connection to 127.18.154.61:41087: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.154.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.61.keytab.
Entry for principal kudu/127.18.154.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.61.keytab.
I20260807 18:15:50.632648 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126328808105-19048-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126328808105-19048-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126328808105-19048-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126328808105-19048-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.154.61:41087
--webserver_interface=127.18.154.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.61.keytab
--principal=kudu/127.18.154.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.154.20:32959
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.154.62:34989,127.18.154.61:41087,127.18.154.60:34071
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126328808105-19048-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126328808105-19048-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126328808105-19048-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 18:15:50.657425 6065 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.154.61" port: 41087 } attempt: 1
I20260807 18:15:50.658747 6068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:50.658231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.61:41087 (local address 127.0.0.1:44682)
0807 18:15:50.658451 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:50.658603 (+ 152us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.61:41087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260807 18:15:50.659946 6068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:50.659540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.61:41087 (local address 127.0.0.1:44690)
0807 18:15:50.659734 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:50.659812 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.61:41087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
W20260807 18:15:50.660141 6065 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.154.61:41087: Network error: Client connection negotiation failed: client connection to 127.18.154.61:41087: connect: Connection refused (error 111)
I20260807 18:15:50.728389 6065 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.154.61" port: 41087 } attempt: 2
I20260807 18:15:50.729728 6068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:50.729130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.61:41087 (local address 127.0.0.1:44706)
0807 18:15:50.729408 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:50.729497 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.61:41087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260807 18:15:50.730809 6068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:50.730442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.61:41087 (local address 127.0.0.1:44720)
0807 18:15:50.730606 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:50.730687 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.61:41087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260807 18:15:50.731081 6065 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.154.61:41087: Network error: Client connection negotiation failed: client connection to 127.18.154.61:41087: connect: Connection refused (error 111)
I20260807 18:15:50.822366 6065 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.154.61" port: 41087 } attempt: 3
I20260807 18:15:50.823647 6068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:50.823188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.61:41087 (local address 127.0.0.1:44736)
0807 18:15:50.823408 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:50.823519 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.61:41087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260807 18:15:50.824805 6068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:50.824461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.61:41087 (local address 127.0.0.1:44750)
0807 18:15:50.824598 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:50.824666 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.61:41087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260807 18:15:50.825095 6065 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.154.61:41087: Network error: Client connection negotiation failed: client connection to 127.18.154.61:41087: connect: Connection refused (error 111)
I20260807 18:15:50.968376 6065 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.154.61" port: 41087 } attempt: 4
I20260807 18:15:50.969856 6068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:50.969086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.61:41087 (local address 127.0.0.1:44760)
0807 18:15:50.969501 (+ 415us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:50.969645 (+ 144us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.61:41087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":235}
I20260807 18:15:50.971027 6068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:50.970580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.61:41087 (local address 127.0.0.1:44776)
0807 18:15:50.970757 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:50.970871 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.61:41087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260807 18:15:50.971297 6065 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.154.61:41087: Network error: Client connection negotiation failed: client connection to 127.18.154.61:41087: connect: Connection refused (error 111)
W20260807 18:15:50.983894 6074 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:15:50.984210 6074 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:15:50.984303 6074 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:15:50.992427 6074 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:15:50.992553 6074 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:15:50.992601 6074 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:15:50.992660 6074 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:15:50.992708 6074 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:15:51.002960 6074 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.154.20:32959
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126328808105-19048-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126328808105-19048-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.18.154.62:34989,127.18.154.61:41087,127.18.154.60:34071
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.61.keytab
--principal=kudu/127.18.154.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.154.61:41087
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126328808105-19048-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.18.154.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126328808105-19048-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:15:51.004397 6074 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:15:51.005878 6074 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:15:51.018424 6083 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:15:51.018517 6085 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:15:51.018563 6082 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:15:51.019308 6074 server_base.cc:1034] running on GCE node
I20260807 18:15:51.020119 6074 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:15:51.021756 6074 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:15:51.023010 6074 hybrid_clock.cc:648] HybridClock initialized: now 1786126551022956 us; error 52 us; skew 500 ppm
I20260807 18:15:51.023407 6074 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:15:51 dist-test-slave-gkw7 krb5kdc[5945](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126551, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:15:51.026896 6074 init.cc:377] Logged in from keytab as kudu/127.18.154.61@KRBTEST.COM (short username kudu)
I20260807 18:15:51.028510 6074 webserver.cc:460] Webserver started at http://127.18.154.61:43737/ using document root <none> and password file <none>
I20260807 18:15:51.029246 6074 fs_manager.cc:359] Metadata directory not provided
I20260807 18:15:51.029376 6074 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:15:51.029752 6074 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:15:51.032841 6074 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126328808105-19048-0/minicluster-data/master-1/data/instance:
uuid: "a686db8af11247cd9a6a069f333d1f0f"
format_stamp: "Formatted at 2026-08-07 18:15:51 on dist-test-slave-gkw7"
server_key: "3fba83d0d502f18d1857df5be50e695d"
server_key_iv: "4c46891d037a6dc7d4abeb934c2fde24"
server_key_version: "encryptionkey@0"
I20260807 18:15:51.033754 6074 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126328808105-19048-0/minicluster-data/master-1/wal/instance:
uuid: "a686db8af11247cd9a6a069f333d1f0f"
format_stamp: "Formatted at 2026-08-07 18:15:51 on dist-test-slave-gkw7"
server_key: "3fba83d0d502f18d1857df5be50e695d"
server_key_iv: "4c46891d037a6dc7d4abeb934c2fde24"
server_key_version: "encryptionkey@0"
I20260807 18:15:51.039083 6074 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260807 18:15:51.042811 6096 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:15:51.044360 6074 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 18:15:51.044560 6074 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126328808105-19048-0/minicluster-data/master-1/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126328808105-19048-0/minicluster-data/master-1/wal
uuid: "a686db8af11247cd9a6a069f333d1f0f"
format_stamp: "Formatted at 2026-08-07 18:15:51 on dist-test-slave-gkw7"
server_key: "3fba83d0d502f18d1857df5be50e695d"
server_key_iv: "4c46891d037a6dc7d4abeb934c2fde24"
server_key_version: "encryptionkey@0"
I20260807 18:15:51.044762 6074 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126328808105-19048-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126328808105-19048-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126328808105-19048-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:15:51.060992 6074 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:15:51.066115 6074 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:15:51.066469 6074 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:15:51.087486 6074 rpc_server.cc:307] RPC server started. Bound to: 127.18.154.61:41087
I20260807 18:15:51.087579 6168 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.154.61:41087 every 8 connection(s)
I20260807 18:15:51.089123 6074 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126328808105-19048-0/minicluster-data/master-1/data/info.pb
I20260807 18:15:51.091989 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 6074
I20260807 18:15:51.092226 19048 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126328808105-19048-0/minicluster-data/master-1/wal/instance
I20260807 18:15:51.092787 19048 external_mini_cluster.cc:1428] Setting key 1590a9faff28dba7327df571cf244377
I20260807 18:15:51.093708 6169 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:15:51.101140 6169 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.154.62" port: 34989 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 18:15:51 dist-test-slave-gkw7 krb5kdc[5945](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126551, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.61@KRBTEST.COM for kudu/127.18.154.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.18.154.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.154.60@KRBTEST.COM" created.
I20260807 18:15:51.129662 6172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:51.104623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59412 (local address 127.18.154.62:34989)
0807 18:15:51.105239 (+ 616us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:51.105247 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:51.106534 (+ 1287us) server_negotiation.cc:407] Connection header received
0807 18:15:51.110085 (+ 3551us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:51.110090 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:51.110172 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:51.110274 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:15:51.112817 (+ 2543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:51.113660 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:51.116561 (+ 2901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:51.117058 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:51.117183 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:51.122342 (+ 5159us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:15:51.122348 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:15:51.122352 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:15:51.122403 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:15:51.125764 (+ 3361us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:15:51.126838 (+ 1074us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:15:51.126846 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:15:51.126852 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:15:51.126992 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:15:51.127493 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:15:51.127500 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:15:51.127504 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:15:51.127926 (+ 422us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:15:51.128107 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0807 18:15:51.128919 (+ 812us) server_negotiation.cc:299] Negotiation successful
0807 18:15:51.129200 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":431,"thread_start_us":290,"threads_started":1}
I20260807 18:15:51.130746 6173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:51.105283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:34989 (local address 127.0.0.1:59412)
0807 18:15:51.106308 (+ 1025us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:51.106365 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0807 18:15:51.109659 (+ 3294us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:15:51.110608 (+ 949us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:15:51.110625 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:15:51.111538 (+ 913us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:15:51.112573 (+ 1035us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:51.112618 (+ 45us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:51.113841 (+ 1223us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:51.113850 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:51.116343 (+ 2493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:51.116364 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:51.117360 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:51.117372 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:51.117801 (+ 429us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:51.118738 (+ 937us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:15:51.118782 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:15:51.122036 (+ 3254us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:15:51.125977 (+ 3941us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:15:51.125989 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:15:51.126009 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:15:51.126696 (+ 687us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:15:51.127130 (+ 434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:15:51.127136 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:15:51.127140 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:15:51.127355 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:15:51.128085 (+ 730us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:15:51.128094 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:15:51.128495 (+ 401us) client_negotiation.cc:769] Sending connection context
0807 18:15:51.128821 (+ 326us) client_negotiation.cc:240] Negotiation successful
0807 18:15:51.129405 (+ 584us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":672,"thread_start_us":426,"threads_started":1}
I20260807 18:15:51.132824 6169 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.154.61" port: 41087 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 18:15:51 dist-test-slave-gkw7 krb5kdc[5945](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126551, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.61@KRBTEST.COM for kudu/127.18.154.61@KRBTEST.COM
I20260807 18:15:51.151665 6173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:51.133698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.61:41087 (local address 127.0.0.1:44790)
0807 18:15:51.133921 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:51.133944 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 18:15:51.134074 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:15:51.135181 (+ 1107us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:15:51.135188 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:15:51.135728 (+ 540us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:15:51.135944 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:51.135958 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:51.137868 (+ 1910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:51.137872 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:51.138695 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:51.138705 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:51.140087 (+ 1382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:51.140091 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:51.140283 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:51.140940 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:15:51.140958 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:15:51.143431 (+ 2473us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:15:51.148590 (+ 5159us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:15:51.148595 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:15:51.148598 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:15:51.149152 (+ 554us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:15:51.149692 (+ 540us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:15:51.149696 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:15:51.149698 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:15:51.149877 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:15:51.150752 (+ 875us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:15:51.150755 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:15:51.150880 (+ 125us) client_negotiation.cc:769] Sending connection context
0807 18:15:51.151001 (+ 121us) client_negotiation.cc:240] Negotiation successful
0807 18:15:51.151377 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":64,"mutex_wait_us":1}
I20260807 18:15:51.152264 6174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:51.133941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44790 (local address 127.18.154.61:41087)
0807 18:15:51.134729 (+ 788us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:51.134736 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:51.134763 (+ 27us) server_negotiation.cc:407] Connection header received
0807 18:15:51.134810 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:51.134817 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:51.135001 (+ 184us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:51.135141 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:15:51.136121 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:51.137695 (+ 1574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:51.138843 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:51.139934 (+ 1091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:51.140093 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:51.143692 (+ 3599us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:15:51.143713 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:15:51.143737 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:15:51.143785 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:15:51.148412 (+ 4627us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:15:51.149305 (+ 893us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:15:51.149319 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:15:51.149326 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:15:51.149517 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:15:51.149976 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:15:51.149985 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:15:51.149992 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:15:51.150561 (+ 569us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:15:51.150806 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0807 18:15:51.151288 (+ 482us) server_negotiation.cc:299] Negotiation successful
0807 18:15:51.151628 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":62,"server-negotiator.queue_time_us":609,"thread_start_us":444,"threads_started":1}
I20260807 18:15:51.154856 6169 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.154.60" port: 34071 } has no permanent_uuid. Determining permanent_uuid...
I20260807 18:15:51.155973 6173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:51.155518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.60:34071 (local address 127.0.0.1:40820)
0807 18:15:51.155724 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:51.155827 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.60:34071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260807 18:15:51.156229 6101 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.154.60:34071: connect: Connection refused (error 111)
I20260807 18:15:51.157994 6173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:51.157612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.60:34071 (local address 127.0.0.1:40832)
0807 18:15:51.157799 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:51.157886 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.60:34071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
W20260807 18:15:51.158281 6169 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.154.60:34071: Network error: Client connection negotiation failed: client connection to 127.18.154.60:34071: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.154.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.60.keytab.
Entry for principal kudu/127.18.154.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.60.keytab.
I20260807 18:15:51.165700 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126328808105-19048-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126328808105-19048-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126328808105-19048-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126328808105-19048-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.154.60:34071
--webserver_interface=127.18.154.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.60.keytab
--principal=kudu/127.18.154.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.154.20:32959
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.154.62:34989,127.18.154.61:41087,127.18.154.60:34071
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126328808105-19048-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126328808105-19048-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126328808105-19048-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 18:15:51.207690 6169 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.154.60" port: 34071 } attempt: 1
I20260807 18:15:51.209057 6173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:51.208533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.60:34071 (local address 127.0.0.1:40836)
0807 18:15:51.208762 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:51.208891 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.60:34071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
I20260807 18:15:51.210270 6173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:51.209889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.60:34071 (local address 127.0.0.1:40838)
0807 18:15:51.210057 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:51.210150 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.60:34071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260807 18:15:51.210546 6169 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.154.60:34071: Network error: Client connection negotiation failed: client connection to 127.18.154.60:34071: connect: Connection refused (error 111)
I20260807 18:15:51.270606 6065 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.154.61" port: 41087 } attempt: 5
I20260807 18:15:51.278743 6169 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.154.60" port: 34071 } attempt: 2
I20260807 18:15:51.280387 6173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:51.279513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.60:34071 (local address 127.0.0.1:40840)
0807 18:15:51.279945 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:51.280115 (+ 170us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.60:34071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":222}
Aug 07 18:15:51 dist-test-slave-gkw7 krb5kdc[5945](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126550, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.62@KRBTEST.COM for kudu/127.18.154.61@KRBTEST.COM
I20260807 18:15:51.281767 6173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:51.281287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.60:34071 (local address 127.0.0.1:40852)
0807 18:15:51.281465 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:51.281573 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.60:34071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260807 18:15:51.282051 6169 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.154.60:34071: Network error: Client connection negotiation failed: client connection to 127.18.154.60:34071: connect: Connection refused (error 111)
I20260807 18:15:51.288203 6174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:51.271668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44792 (local address 127.18.154.61:41087)
0807 18:15:51.271852 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:51.271857 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:51.271904 (+ 47us) server_negotiation.cc:407] Connection header received
0807 18:15:51.272221 (+ 317us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:51.272225 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:51.272295 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:51.272383 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:15:51.273651 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:51.274775 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:51.276975 (+ 2200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:51.277800 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:51.281662 (+ 3862us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:51.282079 (+ 417us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:15:51.282086 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:15:51.282092 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:15:51.282132 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:15:51.285150 (+ 3018us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:15:51.286025 (+ 875us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:15:51.286031 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:15:51.286036 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:15:51.286191 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:15:51.286623 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:15:51.286638 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:15:51.286642 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:15:51.286956 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:15:51.287136 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0807 18:15:51.287606 (+ 470us) server_negotiation.cc:299] Negotiation successful
0807 18:15:51.287854 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260807 18:15:51.288240 6068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:51.271483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.61:41087 (local address 127.0.0.1:44792)
0807 18:15:51.271795 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:51.271826 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0807 18:15:51.272012 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:15:51.272513 (+ 501us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:15:51.272520 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:15:51.273132 (+ 612us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:15:51.273387 (+ 255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:51.273402 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:51.275006 (+ 1604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:51.275032 (+ 26us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:51.276744 (+ 1712us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:51.276767 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:51.278017 (+ 1250us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:51.278024 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:51.278242 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:51.279000 (+ 758us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:15:51.279025 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:15:51.281832 (+ 2807us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:15:51.285336 (+ 3504us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:15:51.285343 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:15:51.285348 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:15:51.285869 (+ 521us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:15:51.286323 (+ 454us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:15:51.286329 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:15:51.286333 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:15:51.286490 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:15:51.287099 (+ 609us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:15:51.287104 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:15:51.287290 (+ 186us) client_negotiation.cc:769] Sending connection context
0807 18:15:51.287485 (+ 195us) client_negotiation.cc:240] Negotiation successful
0807 18:15:51.287825 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":90}
I20260807 18:15:51.289387 6065 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.154.60" port: 34071 } has no permanent_uuid. Determining permanent_uuid...
I20260807 18:15:51.290642 6068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:51.290179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.60:34071 (local address 127.0.0.1:40860)
0807 18:15:51.290368 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:51.290464 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.60:34071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260807 18:15:51.291749 6068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:51.291360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.60:34071 (local address 127.0.0.1:40868)
0807 18:15:51.291520 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:51.291628 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.60:34071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260807 18:15:51.291971 6065 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.154.60:34071: Network error: Client connection negotiation failed: client connection to 127.18.154.60:34071: connect: Connection refused (error 111)
I20260807 18:15:51.343261 6065 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.154.60" port: 34071 } attempt: 1
I20260807 18:15:51.344910 6068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:51.344213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.60:34071 (local address 127.0.0.1:40872)
0807 18:15:51.344514 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:51.344663 (+ 149us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.60:34071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260807 18:15:51.346449 6068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:51.345978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.60:34071 (local address 127.0.0.1:40888)
0807 18:15:51.346232 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:51.346317 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.60:34071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149}
W20260807 18:15:51.346674 6065 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.154.60:34071: Network error: Client connection negotiation failed: client connection to 127.18.154.60:34071: connect: Connection refused (error 111)
I20260807 18:15:51.373312 6169 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.154.60" port: 34071 } attempt: 3
I20260807 18:15:51.374655 6173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:51.374107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.60:34071 (local address 127.0.0.1:40896)
0807 18:15:51.374346 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:51.374470 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.60:34071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
I20260807 18:15:51.375990 6173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:51.375484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.60:34071 (local address 127.0.0.1:40900)
0807 18:15:51.375768 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:51.375869 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.60:34071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":181}
W20260807 18:15:51.376253 6169 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.154.60:34071: Network error: Client connection negotiation failed: client connection to 127.18.154.60:34071: connect: Connection refused (error 111)
I20260807 18:15:51.414939 6065 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.154.60" port: 34071 } attempt: 2
I20260807 18:15:51.416361 6068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:51.415728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.60:34071 (local address 127.0.0.1:40910)
0807 18:15:51.416010 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:51.416142 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.60:34071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
I20260807 18:15:51.417742 6068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:51.417253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.60:34071 (local address 127.0.0.1:40924)
0807 18:15:51.417457 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:51.417573 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.60:34071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260807 18:15:51.418021 6065 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.154.60:34071: Network error: Client connection negotiation failed: client connection to 127.18.154.60:34071: connect: Connection refused (error 111)
I20260807 18:15:51.519497 6169 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.154.60" port: 34071 } attempt: 4
I20260807 18:15:51.520823 6173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:51.520316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.60:34071 (local address 127.0.0.1:40934)
0807 18:15:51.520567 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:51.520669 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.60:34071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260807 18:15:51.521986 6173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:51.521589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.60:34071 (local address 127.0.0.1:40950)
0807 18:15:51.521784 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:51.521866 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.60:34071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
W20260807 18:15:51.522336 6169 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.154.60:34071: Network error: Client connection negotiation failed: client connection to 127.18.154.60:34071: connect: Connection refused (error 111)
I20260807 18:15:51.524350 6065 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.154.60" port: 34071 } attempt: 3
I20260807 18:15:51.525665 6068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:51.524978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.60:34071 (local address 127.0.0.1:40966)
0807 18:15:51.525284 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:51.525374 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.60:34071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162}
I20260807 18:15:51.526908 6068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:51.526451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.60:34071 (local address 127.0.0.1:40970)
0807 18:15:51.526626 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:51.526779 (+ 153us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.60:34071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260807 18:15:51.527161 6065 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.154.60:34071: Network error: Client connection negotiation failed: client connection to 127.18.154.60:34071: connect: Connection refused (error 111)
W20260807 18:15:51.528585 6178 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:15:51.528896 6178 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:15:51.529067 6178 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:15:51.536703 6178 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:15:51.536810 6178 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:15:51.536926 6178 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:15:51.537053 6178 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:15:51.537138 6178 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:15:51.548329 6178 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.154.20:32959
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126328808105-19048-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126328808105-19048-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.18.154.62:34989,127.18.154.61:41087,127.18.154.60:34071
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.60.keytab
--principal=kudu/127.18.154.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.154.60:34071
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126328808105-19048-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.18.154.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126328808105-19048-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:15:51.550077 6178 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:15:51.551573 6178 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:15:51.565218 6184 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:15:51.565267 6183 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:15:51.565380 6186 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:15:51.566277 6178 server_base.cc:1034] running on GCE node
I20260807 18:15:51.567238 6178 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:15:51.569123 6178 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:15:51.570377 6178 hybrid_clock.cc:648] HybridClock initialized: now 1786126551570301 us; error 61 us; skew 500 ppm
I20260807 18:15:51.570799 6178 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:15:51 dist-test-slave-gkw7 krb5kdc[5945](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126551, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:15:51.574937 6178 init.cc:377] Logged in from keytab as kudu/127.18.154.60@KRBTEST.COM (short username kudu)
I20260807 18:15:51.576679 6178 webserver.cc:460] Webserver started at http://127.18.154.60:44733/ using document root <none> and password file <none>
I20260807 18:15:51.577495 6178 fs_manager.cc:359] Metadata directory not provided
I20260807 18:15:51.577724 6178 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:15:51.578094 6178 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:15:51.581518 6178 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126328808105-19048-0/minicluster-data/master-2/data/instance:
uuid: "7e19fa49b73a4330a68a98edff5384e8"
format_stamp: "Formatted at 2026-08-07 18:15:51 on dist-test-slave-gkw7"
server_key: "a211d2cd66d257c26d0de064b04eb886"
server_key_iv: "1c7545167f47f7835ad97e40caa0d531"
server_key_version: "encryptionkey@0"
I20260807 18:15:51.582667 6178 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126328808105-19048-0/minicluster-data/master-2/wal/instance:
uuid: "7e19fa49b73a4330a68a98edff5384e8"
format_stamp: "Formatted at 2026-08-07 18:15:51 on dist-test-slave-gkw7"
server_key: "a211d2cd66d257c26d0de064b04eb886"
server_key_iv: "1c7545167f47f7835ad97e40caa0d531"
server_key_version: "encryptionkey@0"
I20260807 18:15:51.588860 6178 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260807 18:15:51.593060 6195 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:15:51.594736 6178 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:15:51.594957 6178 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126328808105-19048-0/minicluster-data/master-2/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126328808105-19048-0/minicluster-data/master-2/wal
uuid: "7e19fa49b73a4330a68a98edff5384e8"
format_stamp: "Formatted at 2026-08-07 18:15:51 on dist-test-slave-gkw7"
server_key: "a211d2cd66d257c26d0de064b04eb886"
server_key_iv: "1c7545167f47f7835ad97e40caa0d531"
server_key_version: "encryptionkey@0"
I20260807 18:15:51.595217 6178 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126328808105-19048-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126328808105-19048-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126328808105-19048-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:15:51.618603 6178 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:15:51.623914 6178 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:15:51.624321 6178 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:15:51.646412 6178 rpc_server.cc:307] RPC server started. Bound to: 127.18.154.60:34071
I20260807 18:15:51.646447 6268 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.154.60:34071 every 8 connection(s)
I20260807 18:15:51.648102 6178 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126328808105-19048-0/minicluster-data/master-2/data/info.pb
I20260807 18:15:51.648450 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 6178
I20260807 18:15:51.648726 19048 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786126328808105-19048-0/minicluster-data/master-2/wal/instance
I20260807 18:15:51.649510 19048 external_mini_cluster.cc:1428] Setting key 883bf8e74cf87de84727ca4e9a6492ac
I20260807 18:15:51.653256 6271 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:15:51.663426 6271 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.154.62" port: 34989 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 18:15:51 dist-test-slave-gkw7 krb5kdc[5945](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126549, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.154.60@KRBTEST.COM
Aug 07 18:15:51 dist-test-slave-gkw7 krb5kdc[5945](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126551, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.60@KRBTEST.COM for kudu/127.18.154.62@KRBTEST.COM
I20260807 18:15:51.679483 6274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:51.652048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40978 (local address 127.18.154.60:34071)
0807 18:15:51.653164 (+ 1116us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:51.653186 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:51.654399 (+ 1213us) server_negotiation.cc:407] Connection header received
0807 18:15:51.656178 (+ 1779us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:51.656203 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:51.656682 (+ 479us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:51.657049 (+ 367us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:15:51.659381 (+ 2332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:51.661005 (+ 1624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:51.663384 (+ 2379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:51.664177 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:51.664335 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:51.670191 (+ 5856us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:15:51.670201 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:15:51.670213 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:15:51.670253 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:15:51.673654 (+ 3401us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:15:51.674866 (+ 1212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:15:51.674873 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:15:51.674883 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:15:51.675090 (+ 207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:15:51.675597 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:15:51.675601 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:15:51.675603 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:15:51.676233 (+ 630us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:15:51.676690 (+ 457us) server_negotiation.cc:1035] Waiting for connection context
0807 18:15:51.677220 (+ 530us) server_negotiation.cc:299] Negotiation successful
0807 18:15:51.677785 (+ 565us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":816,"thread_start_us":498,"threads_started":1}
I20260807 18:15:51.686271 6277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:51.666858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:34989 (local address 127.0.0.1:59418)
0807 18:15:51.668116 (+ 1258us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:51.668176 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0807 18:15:51.668361 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:15:51.668890 (+ 529us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:15:51.668901 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:15:51.669637 (+ 736us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:15:51.670012 (+ 375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:51.670027 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:51.671177 (+ 1150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:51.671185 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:51.672436 (+ 1251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:51.672450 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:51.673311 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:51.673322 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:51.673520 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:51.674309 (+ 789us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:15:51.674334 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:15:51.677341 (+ 3007us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:15:51.682946 (+ 5605us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:15:51.682957 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:15:51.682965 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:15:51.683689 (+ 724us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:15:51.684222 (+ 533us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:15:51.684229 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:15:51.684236 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:15:51.684430 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:15:51.685152 (+ 722us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:15:51.685159 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:15:51.685349 (+ 190us) client_negotiation.cc:769] Sending connection context
0807 18:15:51.685524 (+ 175us) client_negotiation.cc:240] Negotiation successful
0807 18:15:51.685868 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":948,"spinlock_wait_cycles":27520,"thread_start_us":779,"threads_started":1}
I20260807 18:15:51.686337 6275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:51.666916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59418 (local address 127.18.154.62:34989)
0807 18:15:51.667571 (+ 655us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:51.667591 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:51.668298 (+ 707us) server_negotiation.cc:407] Connection header received
0807 18:15:51.668523 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:51.668533 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:51.668646 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:51.668776 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:15:51.670188 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:51.670970 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:51.672607 (+ 1637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:51.673053 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:51.673254 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:51.677710 (+ 4456us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:15:51.677720 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:15:51.677725 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:15:51.677787 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:15:51.682787 (+ 5000us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:15:51.683851 (+ 1064us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:15:51.683856 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:15:51.683860 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:15:51.684072 (+ 212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:15:51.684589 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:15:51.684595 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:15:51.684598 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:15:51.684994 (+ 396us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:15:51.685188 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0807 18:15:51.685697 (+ 509us) server_negotiation.cc:299] Negotiation successful
0807 18:15:51.685925 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":513,"thread_start_us":384,"threads_started":1}
I20260807 18:15:51.687893 6271 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.154.61" port: 41087 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 18:15:51 dist-test-slave-gkw7 krb5kdc[5945](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126549, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.154.62@KRBTEST.COM
Aug 07 18:15:51 dist-test-slave-gkw7 krb5kdc[5945](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126551, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.60@KRBTEST.COM for kudu/127.18.154.61@KRBTEST.COM
I20260807 18:15:51.701506 6279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:51.683475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59428 (local address 127.18.154.62:34989)
0807 18:15:51.684289 (+ 814us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:51.684295 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:51.684322 (+ 27us) server_negotiation.cc:407] Connection header received
0807 18:15:51.684379 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:51.684385 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:51.684465 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:51.684584 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:15:51.686187 (+ 1603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:51.687597 (+ 1410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:51.689152 (+ 1555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:51.689758 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:51.689934 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:51.694760 (+ 4826us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:15:51.694770 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:15:51.694776 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:15:51.694820 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:15:51.698375 (+ 3555us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:15:51.699169 (+ 794us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:15:51.699175 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:15:51.699183 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:15:51.699336 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:15:51.699816 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:15:51.699821 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:15:51.699825 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:15:51.700216 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:15:51.700414 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0807 18:15:51.700827 (+ 413us) server_negotiation.cc:299] Negotiation successful
0807 18:15:51.701073 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":624,"thread_start_us":339,"threads_started":1}
I20260807 18:15:51.703130 6277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:51.689141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.61:41087 (local address 127.0.0.1:44800)
0807 18:15:51.689344 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:51.689363 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 18:15:51.689458 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:15:51.689863 (+ 405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:15:51.689869 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:15:51.690236 (+ 367us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:15:51.690379 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:51.690387 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:51.691351 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:51.691358 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:51.692229 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:51.692240 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:51.692956 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:51.692962 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:51.693167 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:51.693793 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:15:51.693821 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:15:51.696127 (+ 2306us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:15:51.700171 (+ 4044us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:15:51.700176 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:15:51.700180 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:15:51.700822 (+ 642us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:15:51.701333 (+ 511us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:15:51.701337 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:15:51.701341 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:15:51.701568 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:15:51.702192 (+ 624us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:15:51.702197 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:15:51.702351 (+ 154us) client_negotiation.cc:769] Sending connection context
0807 18:15:51.702500 (+ 149us) client_negotiation.cc:240] Negotiation successful
0807 18:15:51.702795 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":77}
I20260807 18:15:51.703462 6174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:51.689225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44800 (local address 127.18.154.61:41087)
0807 18:15:51.689422 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:51.689428 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:51.689447 (+ 19us) server_negotiation.cc:407] Connection header received
0807 18:15:51.689639 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:51.689650 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:51.689716 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:51.689824 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:15:51.690543 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:51.691207 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:51.692373 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:51.692791 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:51.692927 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:51.696406 (+ 3479us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:15:51.696412 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:15:51.696416 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:15:51.696453 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:15:51.699929 (+ 3476us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:15:51.700977 (+ 1048us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:15:51.700985 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:15:51.700990 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:15:51.701149 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:15:51.701800 (+ 651us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:15:51.701804 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:15:51.701806 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:15:51.702048 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:15:51.702222 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0807 18:15:51.702605 (+ 383us) server_negotiation.cc:299] Negotiation successful
0807 18:15:51.702852 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260807 18:15:51.704443 6065 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.154.60" port: 34071 } attempt: 4
I20260807 18:15:51.704566 6271 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.154.60" port: 34071 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 18:15:51 dist-test-slave-gkw7 krb5kdc[5945](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126551, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.60@KRBTEST.COM for kudu/127.18.154.60@KRBTEST.COM
Aug 07 18:15:51 dist-test-slave-gkw7 krb5kdc[5945](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126550, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.62@KRBTEST.COM for kudu/127.18.154.60@KRBTEST.COM
Aug 07 18:15:51 dist-test-slave-gkw7 krb5kdc[5945](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126549, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.154.61@KRBTEST.COM
I20260807 18:15:51.722070 6277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:51.705531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.60:34071 (local address 127.0.0.1:41010)
0807 18:15:51.705715 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:51.705851 (+ 136us) client_negotiation.cc:174] Beginning negotiation
0807 18:15:51.705958 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:15:51.706796 (+ 838us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:15:51.706800 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:15:51.707237 (+ 437us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:15:51.707424 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:51.707435 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:51.708980 (+ 1545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:51.708988 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:51.710114 (+ 1126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:51.710129 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:51.711206 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:51.711214 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:51.711481 (+ 267us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:51.712137 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:15:51.712156 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:15:51.714743 (+ 2587us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:15:51.718933 (+ 4190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:15:51.718939 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:15:51.718943 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:15:51.719522 (+ 579us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:15:51.719986 (+ 464us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:15:51.719995 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:15:51.720001 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:15:51.720204 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:15:51.720888 (+ 684us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:15:51.720893 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:15:51.721069 (+ 176us) client_negotiation.cc:769] Sending connection context
0807 18:15:51.721202 (+ 133us) client_negotiation.cc:240] Negotiation successful
0807 18:15:51.721572 (+ 370us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47,"mutex_wait_us":9}
I20260807 18:15:51.722581 6281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:51.705632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41010 (local address 127.18.154.60:34071)
0807 18:15:51.706504 (+ 872us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:51.706510 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:51.706533 (+ 23us) server_negotiation.cc:407] Connection header received
0807 18:15:51.706580 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:51.706584 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:51.706655 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:51.707558 (+ 903us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:15:51.707792 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:51.708852 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:51.710263 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:51.711027 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:51.711276 (+ 249us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:51.715019 (+ 3743us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:15:51.715025 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:15:51.715028 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:15:51.715057 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:15:51.718767 (+ 3710us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:15:51.719682 (+ 915us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:15:51.719689 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:15:51.719693 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:15:51.719841 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:15:51.720354 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:15:51.720359 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:15:51.720363 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:15:51.720754 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:15:51.721646 (+ 892us) server_negotiation.cc:1035] Waiting for connection context
0807 18:15:51.721837 (+ 191us) server_negotiation.cc:299] Negotiation successful
0807 18:15:51.722125 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":88,"server-negotiator.queue_time_us":762,"thread_start_us":554,"threads_started":1}
I20260807 18:15:51.723452 6274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:51.705218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40994 (local address 127.18.154.60:34071)
0807 18:15:51.705655 (+ 437us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:51.705661 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:51.705679 (+ 18us) server_negotiation.cc:407] Connection header received
0807 18:15:51.705742 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:51.705747 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:51.705833 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:51.705958 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:15:51.707095 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:51.708305 (+ 1210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:51.710112 (+ 1807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:51.710999 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:51.711245 (+ 246us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:51.716318 (+ 5073us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:15:51.716326 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:15:51.716337 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:15:51.716387 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:15:51.720166 (+ 3779us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:15:51.720879 (+ 713us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:15:51.720887 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:15:51.720891 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:15:51.721080 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:15:51.721519 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:15:51.721555 (+ 36us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:15:51.721562 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:15:51.722010 (+ 448us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:15:51.722160 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0807 18:15:51.722783 (+ 623us) server_negotiation.cc:299] Negotiation successful
0807 18:15:51.723059 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":148,"server-negotiator.queue_time_us":298,"spinlock_wait_cycles":1792}
I20260807 18:15:51.723428 6068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:51.705028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.60:34071 (local address 127.0.0.1:40994)
0807 18:15:51.705217 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:51.705238 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 18:15:51.705383 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:15:51.706245 (+ 862us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:15:51.706251 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:15:51.706674 (+ 423us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:15:51.706923 (+ 249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:51.706938 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:51.708468 (+ 1530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:51.708475 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:51.709944 (+ 1469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:51.709960 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:51.711819 (+ 1859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:51.711828 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:51.712043 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:51.712636 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:15:51.712674 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:15:51.716012 (+ 3338us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:15:51.720310 (+ 4298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:15:51.720316 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:15:51.720321 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:15:51.720743 (+ 422us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:15:51.721233 (+ 490us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:15:51.721239 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:15:51.721243 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:15:51.721397 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:15:51.722232 (+ 835us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:15:51.722237 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:15:51.722446 (+ 209us) client_negotiation.cc:769] Sending connection context
0807 18:15:51.722651 (+ 205us) client_negotiation.cc:240] Negotiation successful
0807 18:15:51.722975 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59}
I20260807 18:15:51.726310 6280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:51.703527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44812 (local address 127.18.154.61:41087)
0807 18:15:51.704221 (+ 694us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:51.704227 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:51.704255 (+ 28us) server_negotiation.cc:407] Connection header received
0807 18:15:51.704302 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:51.704306 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:51.704381 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:51.704489 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:15:51.706805 (+ 2316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:51.707794 (+ 989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:51.709675 (+ 1881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:51.710344 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:51.710491 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:51.717068 (+ 6577us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:15:51.717077 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:15:51.717083 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:15:51.717407 (+ 324us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:15:51.721137 (+ 3730us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:15:51.722094 (+ 957us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:15:51.722102 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:15:51.722105 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:15:51.722553 (+ 448us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:15:51.723324 (+ 771us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:15:51.723329 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:15:51.723332 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:15:51.724096 (+ 764us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:15:51.724405 (+ 309us) server_negotiation.cc:1035] Waiting for connection context
0807 18:15:51.725648 (+ 1243us) server_negotiation.cc:299] Negotiation successful
0807 18:15:51.725893 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":554,"thread_start_us":364,"threads_started":1}
I20260807 18:15:51.728713 6271 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e19fa49b73a4330a68a98edff5384e8: Bootstrap starting.
I20260807 18:15:51.728715 6065 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 486d3b03a9ee4e8da2514f4845e072a4: Bootstrap starting.
I20260807 18:15:51.732715 6065 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 486d3b03a9ee4e8da2514f4845e072a4: Neither blocks nor log segments found. Creating new log.
I20260807 18:15:51.732751 6271 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7e19fa49b73a4330a68a98edff5384e8: Neither blocks nor log segments found. Creating new log.
I20260807 18:15:51.734139 6065 log.cc:824] T 00000000000000000000000000000000 P 486d3b03a9ee4e8da2514f4845e072a4: Log is configured to *not* fsync() on all Append() calls
I20260807 18:15:51.734251 6271 log.cc:824] T 00000000000000000000000000000000 P 7e19fa49b73a4330a68a98edff5384e8: Log is configured to *not* fsync() on all Append() calls
I20260807 18:15:51.735556 19048 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:15:51.738000 6065 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 486d3b03a9ee4e8da2514f4845e072a4: No bootstrap required, opened a new log
I20260807 18:15:51.747239 6065 raft_consensus.cc:348] T 00000000000000000000000000000000 P 486d3b03a9ee4e8da2514f4845e072a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "486d3b03a9ee4e8da2514f4845e072a4" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 34989 } } peers { permanent_uuid: "a686db8af11247cd9a6a069f333d1f0f" member_type: VOTER last_known_addr { host: "127.18.154.61" port: 41087 } } peers { permanent_uuid: "7e19fa49b73a4330a68a98edff5384e8" member_type: VOTER last_known_addr { host: "127.18.154.60" port: 34071 } }
I20260807 18:15:51.747615 6065 raft_consensus.cc:374] T 00000000000000000000000000000000 P 486d3b03a9ee4e8da2514f4845e072a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:15:51.747731 6065 raft_consensus.cc:729] T 00000000000000000000000000000000 P 486d3b03a9ee4e8da2514f4845e072a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 486d3b03a9ee4e8da2514f4845e072a4, State: Initialized, Role: FOLLOWER
I20260807 18:15:51.748625 6065 consensus_queue.cc:260] T 00000000000000000000000000000000 P 486d3b03a9ee4e8da2514f4845e072a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "486d3b03a9ee4e8da2514f4845e072a4" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 34989 } } peers { permanent_uuid: "a686db8af11247cd9a6a069f333d1f0f" member_type: VOTER last_known_addr { host: "127.18.154.61" port: 41087 } } peers { permanent_uuid: "7e19fa49b73a4330a68a98edff5384e8" member_type: VOTER last_known_addr { host: "127.18.154.60" port: 34071 } }
I20260807 18:15:51.751000 6271 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e19fa49b73a4330a68a98edff5384e8: No bootstrap required, opened a new log
I20260807 18:15:51.762609 6294 sys_catalog.cc:455] T 00000000000000000000000000000000 P 486d3b03a9ee4e8da2514f4845e072a4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "486d3b03a9ee4e8da2514f4845e072a4" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 34989 } } peers { permanent_uuid: "a686db8af11247cd9a6a069f333d1f0f" member_type: VOTER last_known_addr { host: "127.18.154.61" port: 41087 } } peers { permanent_uuid: "7e19fa49b73a4330a68a98edff5384e8" member_type: VOTER last_known_addr { host: "127.18.154.60" port: 34071 } } }
I20260807 18:15:51.763209 6294 sys_catalog.cc:458] T 00000000000000000000000000000000 P 486d3b03a9ee4e8da2514f4845e072a4 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 18:15:51.763819 6065 sys_catalog.cc:564] T 00000000000000000000000000000000 P 486d3b03a9ee4e8da2514f4845e072a4 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:15:51.765169 6271 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7e19fa49b73a4330a68a98edff5384e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "486d3b03a9ee4e8da2514f4845e072a4" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 34989 } } peers { permanent_uuid: "a686db8af11247cd9a6a069f333d1f0f" member_type: VOTER last_known_addr { host: "127.18.154.61" port: 41087 } } peers { permanent_uuid: "7e19fa49b73a4330a68a98edff5384e8" member_type: VOTER last_known_addr { host: "127.18.154.60" port: 34071 } }
I20260807 18:15:51.765911 6271 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7e19fa49b73a4330a68a98edff5384e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:15:51.766115 6271 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7e19fa49b73a4330a68a98edff5384e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e19fa49b73a4330a68a98edff5384e8, State: Initialized, Role: FOLLOWER
I20260807 18:15:51.767020 6271 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7e19fa49b73a4330a68a98edff5384e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "486d3b03a9ee4e8da2514f4845e072a4" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 34989 } } peers { permanent_uuid: "a686db8af11247cd9a6a069f333d1f0f" member_type: VOTER last_known_addr { host: "127.18.154.61" port: 41087 } } peers { permanent_uuid: "7e19fa49b73a4330a68a98edff5384e8" member_type: VOTER last_known_addr { host: "127.18.154.60" port: 34071 } }
I20260807 18:15:51.774370 6300 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e19fa49b73a4330a68a98edff5384e8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "486d3b03a9ee4e8da2514f4845e072a4" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 34989 } } peers { permanent_uuid: "a686db8af11247cd9a6a069f333d1f0f" member_type: VOTER last_known_addr { host: "127.18.154.61" port: 41087 } } peers { permanent_uuid: "7e19fa49b73a4330a68a98edff5384e8" member_type: VOTER last_known_addr { host: "127.18.154.60" port: 34071 } } }
I20260807 18:15:51.774798 6300 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e19fa49b73a4330a68a98edff5384e8 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 18:15:51.776422 6271 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7e19fa49b73a4330a68a98edff5384e8 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:15:51.787995 6279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:51.741986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59434 (local address 127.18.154.62:34989)
0807 18:15:51.742399 (+ 413us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:51.742406 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:51.762211 (+ 19805us) server_negotiation.cc:407] Connection header received
0807 18:15:51.762813 (+ 602us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:51.762827 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:51.762940 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:51.763048 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:15:51.770451 (+ 7403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:51.771638 (+ 1187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:51.773984 (+ 2346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:51.774651 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:51.774748 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:51.779148 (+ 4400us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:15:51.779158 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:15:51.779166 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:15:51.779220 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:15:51.783602 (+ 4382us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:15:51.784624 (+ 1022us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:15:51.784633 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:15:51.784637 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:15:51.784829 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:15:51.785434 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:15:51.785440 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:15:51.785444 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:15:51.785923 (+ 479us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:15:51.786216 (+ 293us) server_negotiation.cc:1035] Waiting for connection context
0807 18:15:51.786955 (+ 739us) server_negotiation.cc:299] Negotiation successful
0807 18:15:51.787251 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":268}
W20260807 18:15:51.796387 6000 tablet.cc:2369] T 00000000000000000000000000000000 P 486d3b03a9ee4e8da2514f4845e072a4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:15:51.796818 6280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:51.742182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44814 (local address 127.18.154.61:41087)
0807 18:15:51.742382 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:51.742389 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:51.754112 (+ 11723us) server_negotiation.cc:407] Connection header received
0807 18:15:51.755361 (+ 1249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:51.755375 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:51.755513 (+ 138us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:51.755637 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:15:51.765462 (+ 9825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:51.766585 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:51.773837 (+ 7252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:51.774557 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:51.774683 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:51.789139 (+ 14456us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:15:51.789150 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:15:51.789155 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:15:51.789208 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:15:51.792688 (+ 3480us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:15:51.793986 (+ 1298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:15:51.793994 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:15:51.794000 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:15:51.794189 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:15:51.794757 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:15:51.794763 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:15:51.794767 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:15:51.795194 (+ 427us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:15:51.795464 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0807 18:15:51.796160 (+ 696us) server_negotiation.cc:299] Negotiation successful
0807 18:15:51.796442 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260807 18:15:51.796993 6281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:51.754397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41014 (local address 127.18.154.60:34071)
0807 18:15:51.754684 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:51.754695 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:51.764106 (+ 9411us) server_negotiation.cc:407] Connection header received
0807 18:15:51.764461 (+ 355us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:51.764470 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:51.764587 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:51.764676 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:15:51.770908 (+ 6232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:51.771969 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:51.776937 (+ 4968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:51.777817 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:51.777952 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:51.784185 (+ 6233us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:15:51.784195 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:15:51.784203 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:15:51.784262 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:15:51.788395 (+ 4133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:15:51.794247 (+ 5852us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:15:51.794257 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:15:51.794265 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:15:51.794458 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:15:51.794971 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:15:51.794977 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:15:51.794980 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:15:51.795463 (+ 483us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:15:51.795751 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0807 18:15:51.796391 (+ 640us) server_negotiation.cc:299] Negotiation successful
0807 18:15:51.796640 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260807 18:15:51.798826 6300 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7e19fa49b73a4330a68a98edff5384e8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:15:51.799216 6300 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7e19fa49b73a4330a68a98edff5384e8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "486d3b03a9ee4e8da2514f4845e072a4" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 34989 } } peers { permanent_uuid: "a686db8af11247cd9a6a069f333d1f0f" member_type: VOTER last_known_addr { host: "127.18.154.61" port: 41087 } } peers { permanent_uuid: "7e19fa49b73a4330a68a98edff5384e8" member_type: VOTER last_known_addr { host: "127.18.154.60" port: 34071 } }
I20260807 18:15:51.801421 6300 leader_election.cc:288] T 00000000000000000000000000000000 P 7e19fa49b73a4330a68a98edff5384e8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 486d3b03a9ee4e8da2514f4845e072a4 (127.18.154.62:34989), a686db8af11247cd9a6a069f333d1f0f (127.18.154.61:41087)
I20260807 18:15:51.802747 6025 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7e19fa49b73a4330a68a98edff5384e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "486d3b03a9ee4e8da2514f4845e072a4" is_pre_election: true
I20260807 18:15:51.803508 6025 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 486d3b03a9ee4e8da2514f4845e072a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e19fa49b73a4330a68a98edff5384e8 in term 0.
I20260807 18:15:51.804283 6196 leader_election.cc:302] T 00000000000000000000000000000000 P 7e19fa49b73a4330a68a98edff5384e8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 486d3b03a9ee4e8da2514f4845e072a4, 7e19fa49b73a4330a68a98edff5384e8; no voters:
I20260807 18:15:51.804731 6300 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7e19fa49b73a4330a68a98edff5384e8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:15:51.804909 6300 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7e19fa49b73a4330a68a98edff5384e8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:15:51.805027 6300 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7e19fa49b73a4330a68a98edff5384e8 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:15:51.809556 6300 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7e19fa49b73a4330a68a98edff5384e8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "486d3b03a9ee4e8da2514f4845e072a4" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 34989 } } peers { permanent_uuid: "a686db8af11247cd9a6a069f333d1f0f" member_type: VOTER last_known_addr { host: "127.18.154.61" port: 41087 } } peers { permanent_uuid: "7e19fa49b73a4330a68a98edff5384e8" member_type: VOTER last_known_addr { host: "127.18.154.60" port: 34071 } }
I20260807 18:15:51.811142 6025 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7e19fa49b73a4330a68a98edff5384e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "486d3b03a9ee4e8da2514f4845e072a4"
I20260807 18:15:51.811417 6025 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 486d3b03a9ee4e8da2514f4845e072a4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:15:51.811105 6300 leader_election.cc:288] T 00000000000000000000000000000000 P 7e19fa49b73a4330a68a98edff5384e8 [CANDIDATE]: Term 1 election: Requested vote from peers 486d3b03a9ee4e8da2514f4845e072a4 (127.18.154.62:34989), a686db8af11247cd9a6a069f333d1f0f (127.18.154.61:41087)
I20260807 18:15:51.816977 6025 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 486d3b03a9ee4e8da2514f4845e072a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e19fa49b73a4330a68a98edff5384e8 in term 1.
I20260807 18:15:51.817729 6196 leader_election.cc:302] T 00000000000000000000000000000000 P 7e19fa49b73a4330a68a98edff5384e8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 486d3b03a9ee4e8da2514f4845e072a4, 7e19fa49b73a4330a68a98edff5384e8; no voters:
I20260807 18:15:51.818152 6300 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7e19fa49b73a4330a68a98edff5384e8 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:15:51.818941 6300 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7e19fa49b73a4330a68a98edff5384e8 [term 1 LEADER]: Becoming Leader. State: Replica: 7e19fa49b73a4330a68a98edff5384e8, State: Running, Role: LEADER
I20260807 18:15:51.819752 6300 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7e19fa49b73a4330a68a98edff5384e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "486d3b03a9ee4e8da2514f4845e072a4" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 34989 } } peers { permanent_uuid: "a686db8af11247cd9a6a069f333d1f0f" member_type: VOTER last_known_addr { host: "127.18.154.61" port: 41087 } } peers { permanent_uuid: "7e19fa49b73a4330a68a98edff5384e8" member_type: VOTER last_known_addr { host: "127.18.154.60" port: 34071 } }
I20260807 18:15:51.821810 6169 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.154.60" port: 34071 } attempt: 5
Aug 07 18:15:51 dist-test-slave-gkw7 krb5kdc[5945](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126551, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.61@KRBTEST.COM for kudu/127.18.154.60@KRBTEST.COM
I20260807 18:15:51.837913 6322 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e19fa49b73a4330a68a98edff5384e8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7e19fa49b73a4330a68a98edff5384e8. Latest consensus state: current_term: 1 leader_uuid: "7e19fa49b73a4330a68a98edff5384e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "486d3b03a9ee4e8da2514f4845e072a4" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 34989 } } peers { permanent_uuid: "a686db8af11247cd9a6a069f333d1f0f" member_type: VOTER last_known_addr { host: "127.18.154.61" port: 41087 } } peers { permanent_uuid: "7e19fa49b73a4330a68a98edff5384e8" member_type: VOTER last_known_addr { host: "127.18.154.60" port: 34071 } } }
I20260807 18:15:51.838735 6322 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e19fa49b73a4330a68a98edff5384e8 [sys.catalog]: This master's current role is: LEADER
I20260807 18:15:51.844147 6333 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260807 18:15:51.851963 6173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:51.822463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.60:34071 (local address 127.0.0.1:41018)
0807 18:15:51.822920 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:51.822944 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 18:15:51.823082 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:15:51.824910 (+ 1828us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:15:51.824919 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:15:51.825668 (+ 749us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:15:51.825962 (+ 294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:51.825978 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:51.828230 (+ 2252us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:51.828239 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:51.830295 (+ 2056us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:51.830320 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:51.831797 (+ 1477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:51.831804 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:51.832052 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:51.834705 (+ 2653us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:15:51.834739 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:15:51.838753 (+ 4014us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:15:51.847904 (+ 9151us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:15:51.847910 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:15:51.847914 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:15:51.848583 (+ 669us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:15:51.849444 (+ 861us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:15:51.849450 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:15:51.849454 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:15:51.849647 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:15:51.850901 (+ 1254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:15:51.850906 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:15:51.851096 (+ 190us) client_negotiation.cc:769] Sending connection context
0807 18:15:51.851247 (+ 151us) client_negotiation.cc:240] Negotiation successful
0807 18:15:51.851561 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273}
I20260807 18:15:51.854624 6281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:51.823360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41018 (local address 127.18.154.60:34071)
0807 18:15:51.823619 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:51.823629 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:51.823654 (+ 25us) server_negotiation.cc:407] Connection header received
0807 18:15:51.823733 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:51.823741 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:51.823863 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:51.823988 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:15:51.826479 (+ 2491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:51.827997 (+ 1518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:51.830546 (+ 2549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:51.831560 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:51.831766 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:51.839229 (+ 7463us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:15:51.839239 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:15:51.839246 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:15:51.839303 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:15:51.847608 (+ 8305us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:15:51.848869 (+ 1261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:15:51.848875 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:15:51.848879 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:15:51.849235 (+ 356us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:15:51.849858 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:15:51.849862 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:15:51.849899 (+ 37us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:15:51.850701 (+ 802us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:15:51.852616 (+ 1915us) server_negotiation.cc:1035] Waiting for connection context
0807 18:15:51.852897 (+ 281us) server_negotiation.cc:299] Negotiation successful
0807 18:15:51.853364 (+ 467us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260807 18:15:51.849035 5966 master_runner.cc:186] Error getting master registration for 127.18.154.61:41087: OK, instance_id { permanent_uuid: "a686db8af11247cd9a6a069f333d1f0f" instance_seqno: 1786126551081023 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 18:15:51.860899 5966 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 18:15:51.860934 6169 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a686db8af11247cd9a6a069f333d1f0f: Bootstrap starting.
I20260807 18:15:51.865275 6169 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a686db8af11247cd9a6a069f333d1f0f: Neither blocks nor log segments found. Creating new log.
I20260807 18:15:51.866716 6169 log.cc:824] T 00000000000000000000000000000000 P a686db8af11247cd9a6a069f333d1f0f: Log is configured to *not* fsync() on all Append() calls
W20260807 18:15:51.867486 6332 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7e19fa49b73a4330a68a98edff5384e8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 18:15:51.867683 6332 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 18:15:51.867892 6334 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:15:51.870253 6334 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:15:51.876530 6169 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a686db8af11247cd9a6a069f333d1f0f: No bootstrap required, opened a new log
W20260807 18:15:51.876930 6205 tablet.cc:2369] T 00000000000000000000000000000000 P 7e19fa49b73a4330a68a98edff5384e8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:15:51.865384 6178 master_runner.cc:186] Error getting master registration for 127.18.154.61:41087: OK, instance_id { permanent_uuid: "a686db8af11247cd9a6a069f333d1f0f" instance_seqno: 1786126551081023 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 18:15:51.881438 6178 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 18:15:51.888232 6169 raft_consensus.cc:348] T 00000000000000000000000000000000 P a686db8af11247cd9a6a069f333d1f0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "486d3b03a9ee4e8da2514f4845e072a4" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 34989 } } peers { permanent_uuid: "a686db8af11247cd9a6a069f333d1f0f" member_type: VOTER last_known_addr { host: "127.18.154.61" port: 41087 } } peers { permanent_uuid: "7e19fa49b73a4330a68a98edff5384e8" member_type: VOTER last_known_addr { host: "127.18.154.60" port: 34071 } }
I20260807 18:15:51.888677 6169 raft_consensus.cc:374] T 00000000000000000000000000000000 P a686db8af11247cd9a6a069f333d1f0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:15:51.888798 6169 raft_consensus.cc:729] T 00000000000000000000000000000000 P a686db8af11247cd9a6a069f333d1f0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a686db8af11247cd9a6a069f333d1f0f, State: Initialized, Role: FOLLOWER
I20260807 18:15:51.889672 6169 consensus_queue.cc:260] T 00000000000000000000000000000000 P a686db8af11247cd9a6a069f333d1f0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "486d3b03a9ee4e8da2514f4845e072a4" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 34989 } } peers { permanent_uuid: "a686db8af11247cd9a6a069f333d1f0f" member_type: VOTER last_known_addr { host: "127.18.154.61" port: 41087 } } peers { permanent_uuid: "7e19fa49b73a4330a68a98edff5384e8" member_type: VOTER last_known_addr { host: "127.18.154.60" port: 34071 } }
I20260807 18:15:51.890954 6338 sys_catalog.cc:455] T 00000000000000000000000000000000 P a686db8af11247cd9a6a069f333d1f0f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "486d3b03a9ee4e8da2514f4845e072a4" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 34989 } } peers { permanent_uuid: "a686db8af11247cd9a6a069f333d1f0f" member_type: VOTER last_known_addr { host: "127.18.154.61" port: 41087 } } peers { permanent_uuid: "7e19fa49b73a4330a68a98edff5384e8" member_type: VOTER last_known_addr { host: "127.18.154.60" port: 34071 } } }
I20260807 18:15:51.891633 6338 sys_catalog.cc:458] T 00000000000000000000000000000000 P a686db8af11247cd9a6a069f333d1f0f [sys.catalog]: This master's current role is: FOLLOWER
I20260807 18:15:51.895207 6169 sys_catalog.cc:564] T 00000000000000000000000000000000 P a686db8af11247cd9a6a069f333d1f0f [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:15:51.895277 6137 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7e19fa49b73a4330a68a98edff5384e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a686db8af11247cd9a6a069f333d1f0f" is_pre_election: true
I20260807 18:15:51.895224 6136 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7e19fa49b73a4330a68a98edff5384e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a686db8af11247cd9a6a069f333d1f0f"
I20260807 18:15:51.895912 6137 raft_consensus.cc:2454] T 00000000000000000000000000000000 P a686db8af11247cd9a6a069f333d1f0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e19fa49b73a4330a68a98edff5384e8 in term 0.
I20260807 18:15:51.898293 6136 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a686db8af11247cd9a6a069f333d1f0f [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:15:51.902467 6025 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 486d3b03a9ee4e8da2514f4845e072a4 [term 1 FOLLOWER]: Refusing update from remote peer 7e19fa49b73a4330a68a98edff5384e8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:15:51.903901 6322 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 7e19fa49b73a4330a68a98edff5384e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "486d3b03a9ee4e8da2514f4845e072a4" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 34989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:15:51.904161 6136 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a686db8af11247cd9a6a069f333d1f0f [term 1 FOLLOWER]: Refusing update from remote peer 7e19fa49b73a4330a68a98edff5384e8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:15:51.905946 6322 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 7e19fa49b73a4330a68a98edff5384e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a686db8af11247cd9a6a069f333d1f0f" member_type: VOTER last_known_addr { host: "127.18.154.61" port: 41087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:15:51.934130 6294 sys_catalog.cc:455] T 00000000000000000000000000000000 P 486d3b03a9ee4e8da2514f4845e072a4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7e19fa49b73a4330a68a98edff5384e8. Latest consensus state: current_term: 1 leader_uuid: "7e19fa49b73a4330a68a98edff5384e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "486d3b03a9ee4e8da2514f4845e072a4" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 34989 } } peers { permanent_uuid: "a686db8af11247cd9a6a069f333d1f0f" member_type: VOTER last_known_addr { host: "127.18.154.61" port: 41087 } } peers { permanent_uuid: "7e19fa49b73a4330a68a98edff5384e8" member_type: VOTER last_known_addr { host: "127.18.154.60" port: 34071 } } }
I20260807 18:15:51.934618 6294 sys_catalog.cc:458] T 00000000000000000000000000000000 P 486d3b03a9ee4e8da2514f4845e072a4 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 18:15:51.940896 6294 sys_catalog.cc:455] T 00000000000000000000000000000000 P 486d3b03a9ee4e8da2514f4845e072a4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7e19fa49b73a4330a68a98edff5384e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "486d3b03a9ee4e8da2514f4845e072a4" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 34989 } } peers { permanent_uuid: "a686db8af11247cd9a6a069f333d1f0f" member_type: VOTER last_known_addr { host: "127.18.154.61" port: 41087 } } peers { permanent_uuid: "7e19fa49b73a4330a68a98edff5384e8" member_type: VOTER last_known_addr { host: "127.18.154.60" port: 34071 } } }
I20260807 18:15:51.941181 6294 sys_catalog.cc:458] T 00000000000000000000000000000000 P 486d3b03a9ee4e8da2514f4845e072a4 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 18:15:51.946945 6338 sys_catalog.cc:455] T 00000000000000000000000000000000 P a686db8af11247cd9a6a069f333d1f0f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7e19fa49b73a4330a68a98edff5384e8. Latest consensus state: current_term: 1 leader_uuid: "7e19fa49b73a4330a68a98edff5384e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "486d3b03a9ee4e8da2514f4845e072a4" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 34989 } } peers { permanent_uuid: "a686db8af11247cd9a6a069f333d1f0f" member_type: VOTER last_known_addr { host: "127.18.154.61" port: 41087 } } peers { permanent_uuid: "7e19fa49b73a4330a68a98edff5384e8" member_type: VOTER last_known_addr { host: "127.18.154.60" port: 34071 } } }
I20260807 18:15:51.947314 6338 sys_catalog.cc:458] T 00000000000000000000000000000000 P a686db8af11247cd9a6a069f333d1f0f [sys.catalog]: This master's current role is: FOLLOWER
I20260807 18:15:51.947836 6322 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e19fa49b73a4330a68a98edff5384e8 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7e19fa49b73a4330a68a98edff5384e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "486d3b03a9ee4e8da2514f4845e072a4" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 34989 } } peers { permanent_uuid: "a686db8af11247cd9a6a069f333d1f0f" member_type: VOTER last_known_addr { host: "127.18.154.61" port: 41087 } } peers { permanent_uuid: "7e19fa49b73a4330a68a98edff5384e8" member_type: VOTER last_known_addr { host: "127.18.154.60" port: 34071 } } }
I20260807 18:15:51.948127 6322 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e19fa49b73a4330a68a98edff5384e8 [sys.catalog]: This master's current role is: LEADER
I20260807 18:15:51.948872 6334 catalog_manager.cc:1347] Generated new cluster ID: 6745d2be742845f49127c1260a12bfed
I20260807 18:15:51.948967 6334 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:15:51.949435 6300 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e19fa49b73a4330a68a98edff5384e8 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7e19fa49b73a4330a68a98edff5384e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "486d3b03a9ee4e8da2514f4845e072a4" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 34989 } } peers { permanent_uuid: "a686db8af11247cd9a6a069f333d1f0f" member_type: VOTER last_known_addr { host: "127.18.154.61" port: 41087 } } peers { permanent_uuid: "7e19fa49b73a4330a68a98edff5384e8" member_type: VOTER last_known_addr { host: "127.18.154.60" port: 34071 } } }
I20260807 18:15:51.949823 6300 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e19fa49b73a4330a68a98edff5384e8 [sys.catalog]: This master's current role is: LEADER
I20260807 18:15:51.951380 6338 sys_catalog.cc:455] T 00000000000000000000000000000000 P a686db8af11247cd9a6a069f333d1f0f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7e19fa49b73a4330a68a98edff5384e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "486d3b03a9ee4e8da2514f4845e072a4" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 34989 } } peers { permanent_uuid: "a686db8af11247cd9a6a069f333d1f0f" member_type: VOTER last_known_addr { host: "127.18.154.61" port: 41087 } } peers { permanent_uuid: "7e19fa49b73a4330a68a98edff5384e8" member_type: VOTER last_known_addr { host: "127.18.154.60" port: 34071 } } }
I20260807 18:15:51.951619 6338 sys_catalog.cc:458] T 00000000000000000000000000000000 P a686db8af11247cd9a6a069f333d1f0f [sys.catalog]: This master's current role is: FOLLOWER
I20260807 18:15:51.952028 6362 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260807 18:15:51.966564 6334 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:15:51.968565 6334 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:15:51.977414 6334 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7e19fa49b73a4330a68a98edff5384e8: Generated new TSK 0
I20260807 18:15:51.978701 6334 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:15:52.849731 6333 catalog_manager.cc:1259] Loaded cluster ID: 6745d2be742845f49127c1260a12bfed
I20260807 18:15:52.849886 6333 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 486d3b03a9ee4e8da2514f4845e072a4: loading cluster ID for follower catalog manager: success
I20260807 18:15:52.852609 6333 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 486d3b03a9ee4e8da2514f4845e072a4: acquiring CA information for follower catalog manager: success
I20260807 18:15:52.854682 6333 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 486d3b03a9ee4e8da2514f4845e072a4: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 18:15:52.957665 6362 catalog_manager.cc:1259] Loaded cluster ID: 6745d2be742845f49127c1260a12bfed
I20260807 18:15:52.957804 6362 catalog_manager.cc:1552] T 00000000000000000000000000000000 P a686db8af11247cd9a6a069f333d1f0f: loading cluster ID for follower catalog manager: success
I20260807 18:15:52.961987 6362 catalog_manager.cc:1574] T 00000000000000000000000000000000 P a686db8af11247cd9a6a069f333d1f0f: acquiring CA information for follower catalog manager: success
I20260807 18:15:52.964761 6362 catalog_manager.cc:1602] T 00000000000000000000000000000000 P a686db8af11247cd9a6a069f333d1f0f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 18:15:53.069789 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 5966
I20260807 18:15:53.071245 6057 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:15:53.215283 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 5966
I20260807 18:15:53.242270 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 6074
I20260807 18:15:53.243587 6163 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:15:53.358485 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 6074
I20260807 18:15:53.385061 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 6178
I20260807 18:15:53.386265 6263 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:15:53.497061 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 6178
2026-08-07T18:15:53Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (5056 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786126328808105-19048-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:15:53 dist-test-slave-gkw7 krb5kdc[6385](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:15:53 dist-test-slave-gkw7 krb5kdc[6385](info): set up 2 sockets
Aug 07 18:15:53 dist-test-slave-gkw7 krb5kdc[6385](info): commencing operation
krb5kdc: starting...
W20260807 18:15:54.909667 19048 mini_kdc.cc:121] Time spent starting KDC: real 1.362s user 0.000s sys 0.036s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786126328808105-19048-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786126328808105-19048-0/krb5kdc/test-user.keytab.
Aug 07 18:15:55 dist-test-slave-gkw7 krb5kdc[6385](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126555, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:15:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:15:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.154.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.154.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.154.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.62.keytab.
Entry for principal kudu/127.18.154.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.62.keytab.
I20260807 18:15:55.221884 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786126328808105-19048-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786126328808105-19048-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786126328808105-19048-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.154.62:39189
--webserver_interface=127.18.154.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.62.keytab
--principal=kudu/127.18.154.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.154.20:36731
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.154.62:39189
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786126328808105-19048-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786126328808105-19048-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786126328808105-19048-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:15:55.588042 6406 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:15:55.588364 6406 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:15:55.588522 6406 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:15:55.595793 6406 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:15:55.595909 6406 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:15:55.595997 6406 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:15:55.596093 6406 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:15:55.596192 6406 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:15:55.607031 6406 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.154.20:36731
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786126328808105-19048-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786126328808105-19048-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.154.62:39189
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.62.keytab
--principal=kudu/127.18.154.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.154.62:39189
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.154.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786126328808105-19048-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:15:55.608464 6406 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:15:55.609879 6406 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:15:55.622808 6415 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:15:55.622845 6422 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:15:55.622898 6414 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:15:55.623929 6406 server_base.cc:1034] running on GCE node
I20260807 18:15:55.624732 6406 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:15:55.626351 6406 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:15:55.627595 6406 hybrid_clock.cc:648] HybridClock initialized: now 1786126555627550 us; error 38 us; skew 500 ppm
I20260807 18:15:55.627959 6406 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:15:55 dist-test-slave-gkw7 krb5kdc[6385](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126555, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:15:55.638588 6406 init.cc:377] Logged in from keytab as kudu/127.18.154.62@KRBTEST.COM (short username kudu)
I20260807 18:15:55.640293 6406 webserver.cc:460] Webserver started at http://127.18.154.62:44799/ using document root <none> and password file <none>
I20260807 18:15:55.641036 6406 fs_manager.cc:359] Metadata directory not provided
I20260807 18:15:55.641208 6406 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:15:55.641606 6406 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:15:55.645776 6406 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786126328808105-19048-0/minicluster-data/master-0/data/instance:
uuid: "d17ff5d1a0374099b99f8fc442bd6d68"
format_stamp: "Formatted at 2026-08-07 18:15:55 on dist-test-slave-gkw7"
server_key: "54ec67a0088ead89527b3199b83f7ddb"
server_key_iv: "a6d49bbd7c0f3586c64598dba6ecddaa"
server_key_version: "encryptionkey@0"
I20260807 18:15:55.646724 6406 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786126328808105-19048-0/minicluster-data/master-0/wal/instance:
uuid: "d17ff5d1a0374099b99f8fc442bd6d68"
format_stamp: "Formatted at 2026-08-07 18:15:55 on dist-test-slave-gkw7"
server_key: "54ec67a0088ead89527b3199b83f7ddb"
server_key_iv: "a6d49bbd7c0f3586c64598dba6ecddaa"
server_key_version: "encryptionkey@0"
I20260807 18:15:55.652621 6406 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260807 18:15:55.656364 6431 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:15:55.657871 6406 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:15:55.658057 6406 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786126328808105-19048-0/minicluster-data/master-0/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786126328808105-19048-0/minicluster-data/master-0/wal
uuid: "d17ff5d1a0374099b99f8fc442bd6d68"
format_stamp: "Formatted at 2026-08-07 18:15:55 on dist-test-slave-gkw7"
server_key: "54ec67a0088ead89527b3199b83f7ddb"
server_key_iv: "a6d49bbd7c0f3586c64598dba6ecddaa"
server_key_version: "encryptionkey@0"
I20260807 18:15:55.658270 6406 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786126328808105-19048-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786126328808105-19048-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786126328808105-19048-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:15:55.678180 6406 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:15:55.682922 6406 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:15:55.683248 6406 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:15:55.703012 6406 rpc_server.cc:307] RPC server started. Bound to: 127.18.154.62:39189
I20260807 18:15:55.703063 6504 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.154.62:39189 every 8 connection(s)
I20260807 18:15:55.704461 6406 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
I20260807 18:15:55.708650 6505 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:15:55.712378 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 6406
I20260807 18:15:55.712656 19048 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786126328808105-19048-0/minicluster-data/master-0/wal/instance
I20260807 18:15:55.713439 19048 external_mini_cluster.cc:1428] Setting key 7ec64d8a22a487a378511bb3921557f1
I20260807 18:15:55.731668 6505 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d17ff5d1a0374099b99f8fc442bd6d68: Bootstrap starting.
Aug 07 18:15:55 dist-test-slave-gkw7 krb5kdc[6385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126555, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.154.62@KRBTEST.COM
I20260807 18:15:55.736861 6505 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d17ff5d1a0374099b99f8fc442bd6d68: Neither blocks nor log segments found. Creating new log.
I20260807 18:15:55.738519 6505 log.cc:824] T 00000000000000000000000000000000 P d17ff5d1a0374099b99f8fc442bd6d68: Log is configured to *not* fsync() on all Append() calls
I20260807 18:15:55.743443 6509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:55.716351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47748 (local address 127.18.154.62:39189)
0807 18:15:55.717568 (+ 1217us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:55.717588 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:55.717641 (+ 53us) server_negotiation.cc:407] Connection header received
0807 18:15:55.720357 (+ 2716us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:55.720387 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:55.720957 (+ 570us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:55.721520 (+ 563us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:15:55.723513 (+ 1993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:55.724874 (+ 1361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:55.726902 (+ 2028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:55.728325 (+ 1423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:55.728523 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:55.733459 (+ 4936us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:15:55.733473 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:15:55.733492 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:15:55.733575 (+ 83us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:15:55.737522 (+ 3947us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:15:55.738596 (+ 1074us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:15:55.738608 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:15:55.738618 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:15:55.738884 (+ 266us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:15:55.739468 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:15:55.739473 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:15:55.739476 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:15:55.740399 (+ 923us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:15:55.740759 (+ 360us) server_negotiation.cc:1035] Waiting for connection context
0807 18:15:55.741661 (+ 902us) server_negotiation.cc:299] Negotiation successful
0807 18:15:55.742143 (+ 482us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":871,"thread_start_us":510,"threads_started":1}
I20260807 18:15:55.744390 6505 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d17ff5d1a0374099b99f8fc442bd6d68: No bootstrap required, opened a new log
I20260807 18:15:55.748461 19048 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:15:55.757782 6505 raft_consensus.cc:348] T 00000000000000000000000000000000 P d17ff5d1a0374099b99f8fc442bd6d68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d17ff5d1a0374099b99f8fc442bd6d68" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 39189 } }
I20260807 18:15:55.758373 6505 raft_consensus.cc:374] T 00000000000000000000000000000000 P d17ff5d1a0374099b99f8fc442bd6d68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:15:55.758503 6505 raft_consensus.cc:729] T 00000000000000000000000000000000 P d17ff5d1a0374099b99f8fc442bd6d68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d17ff5d1a0374099b99f8fc442bd6d68, State: Initialized, Role: FOLLOWER
I20260807 18:15:55.759339 6505 consensus_queue.cc:260] T 00000000000000000000000000000000 P d17ff5d1a0374099b99f8fc442bd6d68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d17ff5d1a0374099b99f8fc442bd6d68" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 39189 } }
I20260807 18:15:55.759709 6505 raft_consensus.cc:388] T 00000000000000000000000000000000 P d17ff5d1a0374099b99f8fc442bd6d68 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:15:55.759891 6505 raft_consensus.cc:482] T 00000000000000000000000000000000 P d17ff5d1a0374099b99f8fc442bd6d68 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:15:55.760104 6505 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d17ff5d1a0374099b99f8fc442bd6d68 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:15:55.764545 6505 raft_consensus.cc:504] T 00000000000000000000000000000000 P d17ff5d1a0374099b99f8fc442bd6d68 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d17ff5d1a0374099b99f8fc442bd6d68" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 39189 } }
I20260807 18:15:55.765317 6505 leader_election.cc:302] T 00000000000000000000000000000000 P d17ff5d1a0374099b99f8fc442bd6d68 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d17ff5d1a0374099b99f8fc442bd6d68; no voters:
I20260807 18:15:55.766199 6505 leader_election.cc:288] T 00000000000000000000000000000000 P d17ff5d1a0374099b99f8fc442bd6d68 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:15:55.766340 6513 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d17ff5d1a0374099b99f8fc442bd6d68 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:15:55.767113 6513 raft_consensus.cc:686] T 00000000000000000000000000000000 P d17ff5d1a0374099b99f8fc442bd6d68 [term 1 LEADER]: Becoming Leader. State: Replica: d17ff5d1a0374099b99f8fc442bd6d68, State: Running, Role: LEADER
I20260807 18:15:55.767714 6513 consensus_queue.cc:237] T 00000000000000000000000000000000 P d17ff5d1a0374099b99f8fc442bd6d68 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d17ff5d1a0374099b99f8fc442bd6d68" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 39189 } }
I20260807 18:15:55.769004 6505 sys_catalog.cc:564] T 00000000000000000000000000000000 P d17ff5d1a0374099b99f8fc442bd6d68 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:15:55.771097 6515 sys_catalog.cc:455] T 00000000000000000000000000000000 P d17ff5d1a0374099b99f8fc442bd6d68 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d17ff5d1a0374099b99f8fc442bd6d68" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d17ff5d1a0374099b99f8fc442bd6d68" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 39189 } } }
I20260807 18:15:55.771099 6517 sys_catalog.cc:455] T 00000000000000000000000000000000 P d17ff5d1a0374099b99f8fc442bd6d68 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d17ff5d1a0374099b99f8fc442bd6d68. Latest consensus state: current_term: 1 leader_uuid: "d17ff5d1a0374099b99f8fc442bd6d68" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d17ff5d1a0374099b99f8fc442bd6d68" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 39189 } } }
I20260807 18:15:55.771561 6515 sys_catalog.cc:458] T 00000000000000000000000000000000 P d17ff5d1a0374099b99f8fc442bd6d68 [sys.catalog]: This master's current role is: LEADER
I20260807 18:15:55.771637 6517 sys_catalog.cc:458] T 00000000000000000000000000000000 P d17ff5d1a0374099b99f8fc442bd6d68 [sys.catalog]: This master's current role is: LEADER
I20260807 18:15:55.775609 6522 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:15:55.782943 6522 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:15:55.794739 6522 catalog_manager.cc:1347] Generated new cluster ID: ba82ba74944d417790a93b204b077ef2
I20260807 18:15:55.794883 6522 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:15:55.820919 6522 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:15:55.822988 6522 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:15:55.832799 6522 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d17ff5d1a0374099b99f8fc442bd6d68: Generated new TSK 0
I20260807 18:15:55.833918 6522 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:15:55.862389 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 6406
I20260807 18:15:55.863736 6498 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:15:55.961182 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 6406
2026-08-07T18:15:55Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (2458 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:15:56 dist-test-slave-gkw7 krb5kdc[6552](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:15:56 dist-test-slave-gkw7 krb5kdc[6552](info): set up 2 sockets
Aug 07 18:15:56 dist-test-slave-gkw7 krb5kdc[6552](info): commencing operation
krb5kdc: starting...
W20260807 18:15:57.266974 19048 mini_kdc.cc:121] Time spent starting KDC: real 1.260s user 0.003s sys 0.035s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/krb5kdc/test-user.keytab.
Aug 07 18:15:57 dist-test-slave-gkw7 krb5kdc[6552](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126557, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:15:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:15:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.154.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.154.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.154.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.62.keytab.
Entry for principal kudu/127.18.154.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.62.keytab.
I20260807 18:15:57.570382 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.154.62:39535
--webserver_interface=127.18.154.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.62.keytab
--principal=kudu/127.18.154.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.154.20:44041
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.154.62:39535
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:15:57.921674 6574 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:15:57.922067 6574 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:15:57.922245 6574 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:15:57.929454 6574 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:15:57.929641 6574 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:15:57.929752 6574 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:15:57.929860 6574 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:15:57.929939 6574 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:15:57.939780 6574 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.154.20:44041
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.154.62:39535
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.62.keytab
--principal=kudu/127.18.154.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.154.62:39535
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.154.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:15:57.941212 6574 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:15:57.942654 6574 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:15:57.955142 6580 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:15:57.955081 6581 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:15:57.955111 6584 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:15:57.956091 6574 server_base.cc:1034] running on GCE node
I20260807 18:15:57.956880 6574 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:15:57.958442 6574 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:15:57.959666 6574 hybrid_clock.cc:648] HybridClock initialized: now 1786126557959610 us; error 39 us; skew 500 ppm
I20260807 18:15:57.960069 6574 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:15:57 dist-test-slave-gkw7 krb5kdc[6552](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126557, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:15:57.963477 6574 init.cc:377] Logged in from keytab as kudu/127.18.154.62@KRBTEST.COM (short username kudu)
I20260807 18:15:57.965119 6574 webserver.cc:460] Webserver started at http://127.18.154.62:36013/ using document root <none> and password file <none>
I20260807 18:15:57.965932 6574 fs_manager.cc:359] Metadata directory not provided
I20260807 18:15:57.966085 6574 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:15:57.966398 6574 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:15:57.969471 6574 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/master-0/data/instance:
uuid: "d800f8a66628417d808b2e4b0961b702"
format_stamp: "Formatted at 2026-08-07 18:15:57 on dist-test-slave-gkw7"
server_key: "9b0188e763314f93735051427425d348"
server_key_iv: "9e7ec036ea217b2c0de08e7ac782e263"
server_key_version: "encryptionkey@0"
I20260807 18:15:57.970335 6574 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/master-0/wal/instance:
uuid: "d800f8a66628417d808b2e4b0961b702"
format_stamp: "Formatted at 2026-08-07 18:15:57 on dist-test-slave-gkw7"
server_key: "9b0188e763314f93735051427425d348"
server_key_iv: "9e7ec036ea217b2c0de08e7ac782e263"
server_key_version: "encryptionkey@0"
I20260807 18:15:57.975721 6574 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 18:15:57.979473 6594 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:15:57.980876 6574 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 18:15:57.981070 6574 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/master-0/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/master-0/wal
uuid: "d800f8a66628417d808b2e4b0961b702"
format_stamp: "Formatted at 2026-08-07 18:15:57 on dist-test-slave-gkw7"
server_key: "9b0188e763314f93735051427425d348"
server_key_iv: "9e7ec036ea217b2c0de08e7ac782e263"
server_key_version: "encryptionkey@0"
I20260807 18:15:57.981272 6574 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:15:57.996089 6574 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:15:58.000618 6574 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:15:58.000929 6574 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:15:58.021636 6574 rpc_server.cc:307] RPC server started. Bound to: 127.18.154.62:39535
I20260807 18:15:58.021696 6668 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.154.62:39535 every 8 connection(s)
I20260807 18:15:58.023193 6574 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
I20260807 18:15:58.027355 6669 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:15:58.030786 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 6574
I20260807 18:15:58.031059 19048 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/master-0/wal/instance
I20260807 18:15:58.032028 19048 external_mini_cluster.cc:1428] Setting key b12ba2cd491b65b9597a7b685e0ff962
I20260807 18:15:58.042110 6669 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d800f8a66628417d808b2e4b0961b702: Bootstrap starting.
I20260807 18:15:58.045643 6669 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d800f8a66628417d808b2e4b0961b702: Neither blocks nor log segments found. Creating new log.
I20260807 18:15:58.046768 6669 log.cc:824] T 00000000000000000000000000000000 P d800f8a66628417d808b2e4b0961b702: Log is configured to *not* fsync() on all Append() calls
Aug 07 18:15:58 dist-test-slave-gkw7 krb5kdc[6552](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126557, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.154.62@KRBTEST.COM
I20260807 18:15:58.050390 6669 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d800f8a66628417d808b2e4b0961b702: No bootstrap required, opened a new log
I20260807 18:15:58.058420 6675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:58.034476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50028 (local address 127.18.154.62:39535)
0807 18:15:58.035317 (+ 841us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:58.035330 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:58.035375 (+ 45us) server_negotiation.cc:407] Connection header received
0807 18:15:58.037107 (+ 1732us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:58.037132 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:58.037635 (+ 503us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:58.038052 (+ 417us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:15:58.040109 (+ 2057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:58.041465 (+ 1356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:58.043087 (+ 1622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:58.043810 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:58.044017 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:58.048871 (+ 4854us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:15:58.048882 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:15:58.048902 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:15:58.048950 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:15:58.053260 (+ 4310us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:15:58.054132 (+ 872us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:15:58.054142 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:15:58.054151 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:15:58.054388 (+ 237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:15:58.054816 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:15:58.054820 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:15:58.054823 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:15:58.055581 (+ 758us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:15:58.055932 (+ 351us) server_negotiation.cc:1035] Waiting for connection context
0807 18:15:58.056349 (+ 417us) server_negotiation.cc:299] Negotiation successful
0807 18:15:58.056869 (+ 520us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":620,"thread_start_us":401,"threads_started":1}
I20260807 18:15:58.061403 6669 raft_consensus.cc:348] T 00000000000000000000000000000000 P d800f8a66628417d808b2e4b0961b702 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d800f8a66628417d808b2e4b0961b702" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 39535 } }
I20260807 18:15:58.061964 6669 raft_consensus.cc:374] T 00000000000000000000000000000000 P d800f8a66628417d808b2e4b0961b702 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:15:58.062083 6669 raft_consensus.cc:729] T 00000000000000000000000000000000 P d800f8a66628417d808b2e4b0961b702 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d800f8a66628417d808b2e4b0961b702, State: Initialized, Role: FOLLOWER
I20260807 18:15:58.062701 6669 consensus_queue.cc:260] T 00000000000000000000000000000000 P d800f8a66628417d808b2e4b0961b702 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d800f8a66628417d808b2e4b0961b702" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 39535 } }
I20260807 18:15:58.062934 6669 raft_consensus.cc:388] T 00000000000000000000000000000000 P d800f8a66628417d808b2e4b0961b702 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:15:58.063089 6669 raft_consensus.cc:482] T 00000000000000000000000000000000 P d800f8a66628417d808b2e4b0961b702 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:15:58.063274 6669 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d800f8a66628417d808b2e4b0961b702 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:15:58.065575 6669 raft_consensus.cc:504] T 00000000000000000000000000000000 P d800f8a66628417d808b2e4b0961b702 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d800f8a66628417d808b2e4b0961b702" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 39535 } }
I20260807 18:15:58.066243 6669 leader_election.cc:302] T 00000000000000000000000000000000 P d800f8a66628417d808b2e4b0961b702 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d800f8a66628417d808b2e4b0961b702; no voters:
I20260807 18:15:58.067021 6669 leader_election.cc:288] T 00000000000000000000000000000000 P d800f8a66628417d808b2e4b0961b702 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:15:58.067188 6677 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d800f8a66628417d808b2e4b0961b702 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:15:58.068499 6677 raft_consensus.cc:686] T 00000000000000000000000000000000 P d800f8a66628417d808b2e4b0961b702 [term 1 LEADER]: Becoming Leader. State: Replica: d800f8a66628417d808b2e4b0961b702, State: Running, Role: LEADER
I20260807 18:15:58.069212 6669 sys_catalog.cc:564] T 00000000000000000000000000000000 P d800f8a66628417d808b2e4b0961b702 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:15:58.069090 6677 consensus_queue.cc:237] T 00000000000000000000000000000000 P d800f8a66628417d808b2e4b0961b702 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d800f8a66628417d808b2e4b0961b702" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 39535 } }
I20260807 18:15:58.075641 6678 sys_catalog.cc:455] T 00000000000000000000000000000000 P d800f8a66628417d808b2e4b0961b702 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d800f8a66628417d808b2e4b0961b702" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d800f8a66628417d808b2e4b0961b702" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 39535 } } }
I20260807 18:15:58.076148 6678 sys_catalog.cc:458] T 00000000000000000000000000000000 P d800f8a66628417d808b2e4b0961b702 [sys.catalog]: This master's current role is: LEADER
I20260807 18:15:58.076615 6679 sys_catalog.cc:455] T 00000000000000000000000000000000 P d800f8a66628417d808b2e4b0961b702 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d800f8a66628417d808b2e4b0961b702. Latest consensus state: current_term: 1 leader_uuid: "d800f8a66628417d808b2e4b0961b702" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d800f8a66628417d808b2e4b0961b702" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 39535 } } }
I20260807 18:15:58.076948 6679 sys_catalog.cc:458] T 00000000000000000000000000000000 P d800f8a66628417d808b2e4b0961b702 [sys.catalog]: This master's current role is: LEADER
I20260807 18:15:58.077142 6694 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:15:58.082578 6694 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:15:58.093077 6694 catalog_manager.cc:1347] Generated new cluster ID: 67ef141743fc47daa7707f2f49ced5ca
I20260807 18:15:58.093248 6694 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:15:58.129060 6694 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:15:58.130694 6694 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:15:58.141001 6694 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d800f8a66628417d808b2e4b0961b702: Generated new TSK 0
I20260807 18:15:58.142078 6694 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.154.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.154.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.154.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.1.keytab.
Entry for principal kudu/127.18.154.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.1.keytab.
I20260807 18:15:58.221400 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.154.1:0
--local_ip_for_outbound_sockets=127.18.154.1
--webserver_interface=127.18.154.1
--webserver_port=0
--tserver_master_addrs=127.18.154.62:39535
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.1.keytab
--principal=kudu/127.18.154.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.154.20:44041
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:15:58.585031 6703 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:15:58.585312 6703 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:15:58.585466 6703 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:15:58.592833 6703 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:15:58.592941 6703 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:15:58.593113 6703 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.154.1
I20260807 18:15:58.602843 6703 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.154.20:44041
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.1.keytab
--principal=kudu/127.18.154.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.154.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.154.1
--webserver_port=0
--tserver_master_addrs=127.18.154.62:39535
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.154.1
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:15:58.604233 6703 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:15:58.605621 6703 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:15:58.619005 6711 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:15:58.618960 6712 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:15:58.619061 6715 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:15:58.620224 6703 server_base.cc:1034] running on GCE node
I20260807 18:15:58.620888 6703 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:15:58.621971 6703 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:15:58.623206 6703 hybrid_clock.cc:648] HybridClock initialized: now 1786126558623155 us; error 42 us; skew 500 ppm
I20260807 18:15:58.623525 6703 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:15:58 dist-test-slave-gkw7 krb5kdc[6552](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126558, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:15:58.626884 6703 init.cc:377] Logged in from keytab as kudu/127.18.154.1@KRBTEST.COM (short username kudu)
I20260807 18:15:58.628407 6703 webserver.cc:460] Webserver started at http://127.18.154.1:36253/ using document root <none> and password file <none>
I20260807 18:15:58.629117 6703 fs_manager.cc:359] Metadata directory not provided
I20260807 18:15:58.629256 6703 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:15:58.629644 6703 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:15:58.632577 6703 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/ts-0/data/instance:
uuid: "70281fa69b7949aead7a99ec552fa256"
format_stamp: "Formatted at 2026-08-07 18:15:58 on dist-test-slave-gkw7"
server_key: "e9b0db28c63883e21bb2301158686274"
server_key_iv: "c890fb8f75d03d6d75dcf07022b40aae"
server_key_version: "encryptionkey@0"
I20260807 18:15:58.633378 6703 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/ts-0/wal/instance:
uuid: "70281fa69b7949aead7a99ec552fa256"
format_stamp: "Formatted at 2026-08-07 18:15:58 on dist-test-slave-gkw7"
server_key: "e9b0db28c63883e21bb2301158686274"
server_key_iv: "c890fb8f75d03d6d75dcf07022b40aae"
server_key_version: "encryptionkey@0"
I20260807 18:15:58.638664 6703 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260807 18:15:58.642338 6725 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:15:58.643685 6703 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 18:15:58.643872 6703 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/ts-0/wal
uuid: "70281fa69b7949aead7a99ec552fa256"
format_stamp: "Formatted at 2026-08-07 18:15:58 on dist-test-slave-gkw7"
server_key: "e9b0db28c63883e21bb2301158686274"
server_key_iv: "c890fb8f75d03d6d75dcf07022b40aae"
server_key_version: "encryptionkey@0"
I20260807 18:15:58.644088 6703 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:15:58.663882 6703 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:15:58.668397 6703 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:15:58.668711 6703 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:15:58.669847 6703 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:15:58.671455 6703 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:15:58.671556 6703 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:15:58.671703 6703 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:15:58.671774 6703 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:15:58.706704 6703 rpc_server.cc:307] RPC server started. Bound to: 127.18.154.1:44921
I20260807 18:15:58.706727 6892 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.154.1:44921 every 8 connection(s)
I20260807 18:15:58.708146 6703 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/ts-0/data/info.pb
I20260807 18:15:58.711542 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 6703
I20260807 18:15:58.711778 19048 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/ts-0/wal/instance
I20260807 18:15:58.712460 19048 external_mini_cluster.cc:1428] Setting key c39af102ec12a9c831981a3b7242485e
Aug 07 18:15:58 dist-test-slave-gkw7 krb5kdc[6552](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126558, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.1@KRBTEST.COM for kudu/127.18.154.62@KRBTEST.COM
I20260807 18:15:58.733202 6895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:58.710926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.1:45361 (local address 127.18.154.62:39535)
0807 18:15:58.711457 (+ 531us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:58.711465 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:58.712751 (+ 1286us) server_negotiation.cc:407] Connection header received
0807 18:15:58.715395 (+ 2644us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:58.715400 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:58.715467 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:58.715554 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:15:58.717968 (+ 2414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:58.718731 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:58.721353 (+ 2622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:58.721892 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:58.722004 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:58.726495 (+ 4491us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:15:58.726500 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:15:58.726503 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:15:58.726539 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:15:58.729696 (+ 3157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:15:58.730709 (+ 1013us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:15:58.730713 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:15:58.730716 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:15:58.730842 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:15:58.731366 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:15:58.731370 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:15:58.731372 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:15:58.731643 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:15:58.731773 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0807 18:15:58.732748 (+ 975us) server_negotiation.cc:299] Negotiation successful
0807 18:15:58.732931 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":372,"thread_start_us":237,"threads_started":1}
I20260807 18:15:58.734848 6896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:58.711524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:39535 (local address 127.18.154.1:45361)
0807 18:15:58.712539 (+ 1015us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:58.712592 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0807 18:15:58.714899 (+ 2307us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:15:58.715913 (+ 1014us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:15:58.715928 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:15:58.716800 (+ 872us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:15:58.717742 (+ 942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:58.717774 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:58.718905 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:58.718913 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:58.721142 (+ 2229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:58.721161 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:58.722163 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:58.722173 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:58.722541 (+ 368us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:58.723339 (+ 798us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:15:58.723397 (+ 58us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:15:58.726235 (+ 2838us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:15:58.729846 (+ 3611us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:15:58.729858 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:15:58.729881 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:15:58.730547 (+ 666us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:15:58.730957 (+ 410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:15:58.730965 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:15:58.730971 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:15:58.731240 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:15:58.731799 (+ 559us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:15:58.731810 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:15:58.732299 (+ 489us) client_negotiation.cc:769] Sending connection context
0807 18:15:58.732650 (+ 351us) client_negotiation.cc:240] Negotiation successful
0807 18:15:58.733282 (+ 632us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":737,"thread_start_us":415,"threads_started":1}
I20260807 18:15:58.736680 6893 heartbeater.cc:347] Connected to a master server at 127.18.154.62:39535
I20260807 18:15:58.737022 6893 heartbeater.cc:464] Registering TS with master...
I20260807 18:15:58.737900 6893 heartbeater.cc:511] Master 127.18.154.62:39535 requested a full tablet report, sending...
I20260807 18:15:58.740419 6618 ts_manager.cc:194] Registered new tserver with Master: 70281fa69b7949aead7a99ec552fa256 (127.18.154.1:44921)
I20260807 18:15:58.742748 6618 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.154.1@KRBTEST.COM'} at 127.18.154.1:45361
WARNING: no policy specified for kudu/127.18.154.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.154.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.154.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.2.keytab.
Entry for principal kudu/127.18.154.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.2.keytab.
I20260807 18:15:58.785264 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.154.2:0
--local_ip_for_outbound_sockets=127.18.154.2
--webserver_interface=127.18.154.2
--webserver_port=0
--tserver_master_addrs=127.18.154.62:39535
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.2.keytab
--principal=kudu/127.18.154.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.154.20:44041
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:15:59.131556 6899 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:15:59.131837 6899 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:15:59.131997 6899 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:15:59.139413 6899 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:15:59.139544 6899 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:15:59.139734 6899 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.154.2
I20260807 18:15:59.150005 6899 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.154.20:44041
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.2.keytab
--principal=kudu/127.18.154.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.154.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.154.2
--webserver_port=0
--tserver_master_addrs=127.18.154.62:39535
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.154.2
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:15:59.151391 6899 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:15:59.152786 6899 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:15:59.166922 6910 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:15:59.166971 6908 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:15:59.166993 6907 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:15:59.168258 6899 server_base.cc:1034] running on GCE node
I20260807 18:15:59.168978 6899 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:15:59.170163 6899 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:15:59.171375 6899 hybrid_clock.cc:648] HybridClock initialized: now 1786126559171310 us; error 56 us; skew 500 ppm
I20260807 18:15:59.171749 6899 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:15:59 dist-test-slave-gkw7 krb5kdc[6552](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126559, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:15:59.175571 6899 init.cc:377] Logged in from keytab as kudu/127.18.154.2@KRBTEST.COM (short username kudu)
I20260807 18:15:59.177242 6899 webserver.cc:460] Webserver started at http://127.18.154.2:35589/ using document root <none> and password file <none>
I20260807 18:15:59.178081 6899 fs_manager.cc:359] Metadata directory not provided
I20260807 18:15:59.178241 6899 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:15:59.178601 6899 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:15:59.181803 6899 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/ts-1/data/instance:
uuid: "20725a0434e54ab9b7d73c8733499c30"
format_stamp: "Formatted at 2026-08-07 18:15:59 on dist-test-slave-gkw7"
server_key: "7b63ade0bc23bcb7a97748847d6b3f5a"
server_key_iv: "fec9806a8bfee1cbc4c4db5873e15e93"
server_key_version: "encryptionkey@0"
I20260807 18:15:59.182679 6899 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/ts-1/wal/instance:
uuid: "20725a0434e54ab9b7d73c8733499c30"
format_stamp: "Formatted at 2026-08-07 18:15:59 on dist-test-slave-gkw7"
server_key: "7b63ade0bc23bcb7a97748847d6b3f5a"
server_key_iv: "fec9806a8bfee1cbc4c4db5873e15e93"
server_key_version: "encryptionkey@0"
I20260807 18:15:59.189252 6899 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260807 18:15:59.193038 6919 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:15:59.194707 6899 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:15:59.194914 6899 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/ts-1/wal
uuid: "20725a0434e54ab9b7d73c8733499c30"
format_stamp: "Formatted at 2026-08-07 18:15:59 on dist-test-slave-gkw7"
server_key: "7b63ade0bc23bcb7a97748847d6b3f5a"
server_key_iv: "fec9806a8bfee1cbc4c4db5873e15e93"
server_key_version: "encryptionkey@0"
I20260807 18:15:59.195160 6899 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:15:59.232275 6899 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:15:59.237066 6899 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:15:59.237429 6899 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:15:59.238504 6899 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:15:59.240011 6899 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:15:59.240123 6899 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:15:59.240316 6899 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:15:59.240396 6899 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:15:59.281922 6899 rpc_server.cc:307] RPC server started. Bound to: 127.18.154.2:45587
I20260807 18:15:59.281945 7066 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.154.2:45587 every 8 connection(s)
I20260807 18:15:59.283632 6899 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/ts-1/data/info.pb
I20260807 18:15:59.286262 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 6899
I20260807 18:15:59.286465 19048 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/ts-1/wal/instance
I20260807 18:15:59.287015 19048 external_mini_cluster.cc:1428] Setting key 514987ca9609969d835d62ae57411570
Aug 07 18:15:59 dist-test-slave-gkw7 krb5kdc[6552](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126559, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.2@KRBTEST.COM for kudu/127.18.154.62@KRBTEST.COM
I20260807 18:15:59.312393 7071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:59.286739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.2:58217 (local address 127.18.154.62:39535)
0807 18:15:59.287281 (+ 542us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:59.287288 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:59.288769 (+ 1481us) server_negotiation.cc:407] Connection header received
0807 18:15:59.291806 (+ 3037us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:59.291811 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:59.291876 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:59.291969 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:15:59.294410 (+ 2441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:59.295151 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:59.298112 (+ 2961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:59.298548 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:59.298644 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:59.303984 (+ 5340us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:15:59.303990 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:15:59.303994 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:15:59.304041 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:15:59.308477 (+ 4436us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:15:59.309316 (+ 839us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:15:59.309325 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:15:59.309333 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:15:59.309511 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:15:59.310062 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:15:59.310071 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:15:59.310079 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:15:59.310524 (+ 445us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:15:59.310698 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0807 18:15:59.311510 (+ 812us) server_negotiation.cc:299] Negotiation successful
0807 18:15:59.311806 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":384,"thread_start_us":255,"threads_started":1}
I20260807 18:15:59.313086 7072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:59.287511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:39535 (local address 127.18.154.2:58217)
0807 18:15:59.288570 (+ 1059us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:59.288629 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0807 18:15:59.291440 (+ 2811us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:15:59.292318 (+ 878us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:15:59.292334 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:15:59.293223 (+ 889us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:15:59.294214 (+ 991us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:59.294246 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:59.295324 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:59.295332 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:59.297934 (+ 2602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:59.297953 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:59.298830 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:59.298842 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:59.299287 (+ 445us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:59.300181 (+ 894us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:15:59.300229 (+ 48us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:15:59.303623 (+ 3394us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:15:59.308652 (+ 5029us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:15:59.308661 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:15:59.308676 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:15:59.309153 (+ 477us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:15:59.309686 (+ 533us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:15:59.309690 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:15:59.309692 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:15:59.309917 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:15:59.310678 (+ 761us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:15:59.310685 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:15:59.311059 (+ 374us) client_negotiation.cc:769] Sending connection context
0807 18:15:59.311339 (+ 280us) client_negotiation.cc:240] Negotiation successful
0807 18:15:59.311799 (+ 460us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":737,"thread_start_us":485,"threads_started":1}
I20260807 18:15:59.314926 7069 heartbeater.cc:347] Connected to a master server at 127.18.154.62:39535
I20260807 18:15:59.315282 7069 heartbeater.cc:464] Registering TS with master...
I20260807 18:15:59.316143 7069 heartbeater.cc:511] Master 127.18.154.62:39535 requested a full tablet report, sending...
I20260807 18:15:59.317965 6618 ts_manager.cc:194] Registered new tserver with Master: 20725a0434e54ab9b7d73c8733499c30 (127.18.154.2:45587)
I20260807 18:15:59.318879 6618 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.154.2@KRBTEST.COM'} at 127.18.154.2:58217
WARNING: no policy specified for kudu/127.18.154.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.154.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.154.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.3.keytab.
Entry for principal kudu/127.18.154.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.3.keytab.
I20260807 18:15:59.361054 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.154.3:0
--local_ip_for_outbound_sockets=127.18.154.3
--webserver_interface=127.18.154.3
--webserver_port=0
--tserver_master_addrs=127.18.154.62:39535
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.3.keytab
--principal=kudu/127.18.154.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.154.20:44041
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:15:59.715267 7076 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:15:59.715548 7076 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:15:59.715713 7076 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:15:59.723155 7076 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:15:59.723276 7076 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:15:59.723459 7076 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.154.3
I20260807 18:15:59.733562 7076 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.154.20:44041
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.3.keytab
--principal=kudu/127.18.154.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.154.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.154.3
--webserver_port=0
--tserver_master_addrs=127.18.154.62:39535
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.154.3
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:15:59.735057 7076 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:15:59.736475 7076 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 18:15:59.746236 6893 heartbeater.cc:503] Master 127.18.154.62:39535 was elected leader, sending a full tablet report...
W20260807 18:15:59.751348 7087 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:15:59.751312 7093 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:15:59.751259 7091 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:15:59.752285 7076 server_base.cc:1034] running on GCE node
I20260807 18:15:59.753048 7076 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:15:59.754182 7076 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:15:59.755465 7076 hybrid_clock.cc:648] HybridClock initialized: now 1786126559755409 us; error 63 us; skew 500 ppm
I20260807 18:15:59.755847 7076 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:15:59 dist-test-slave-gkw7 krb5kdc[6552](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126559, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:15:59.759644 7076 init.cc:377] Logged in from keytab as kudu/127.18.154.3@KRBTEST.COM (short username kudu)
I20260807 18:15:59.761263 7076 webserver.cc:460] Webserver started at http://127.18.154.3:38245/ using document root <none> and password file <none>
I20260807 18:15:59.762127 7076 fs_manager.cc:359] Metadata directory not provided
I20260807 18:15:59.762244 7076 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:15:59.762516 7076 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:15:59.765784 7076 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/ts-2/data/instance:
uuid: "9fc0d7d72d654b3fbed6817c5715dc59"
format_stamp: "Formatted at 2026-08-07 18:15:59 on dist-test-slave-gkw7"
server_key: "144ea1a0532ead184ac0c8c2989ee2a9"
server_key_iv: "ed37e496730df29551cea2c777ddc950"
server_key_version: "encryptionkey@0"
I20260807 18:15:59.766635 7076 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/ts-2/wal/instance:
uuid: "9fc0d7d72d654b3fbed6817c5715dc59"
format_stamp: "Formatted at 2026-08-07 18:15:59 on dist-test-slave-gkw7"
server_key: "144ea1a0532ead184ac0c8c2989ee2a9"
server_key_iv: "ed37e496730df29551cea2c777ddc950"
server_key_version: "encryptionkey@0"
I20260807 18:15:59.772706 7076 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260807 18:15:59.776760 7103 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:15:59.778468 7076 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:15:59.778613 7076 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/ts-2/wal
uuid: "9fc0d7d72d654b3fbed6817c5715dc59"
format_stamp: "Formatted at 2026-08-07 18:15:59 on dist-test-slave-gkw7"
server_key: "144ea1a0532ead184ac0c8c2989ee2a9"
server_key_iv: "ed37e496730df29551cea2c777ddc950"
server_key_version: "encryptionkey@0"
I20260807 18:15:59.778796 7076 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:15:59.796499 7076 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:15:59.802269 7076 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:15:59.802543 7076 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:15:59.803535 7076 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:15:59.804961 7076 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:15:59.805035 7076 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:15:59.805130 7076 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:15:59.805178 7076 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:15:59.841212 7076 rpc_server.cc:307] RPC server started. Bound to: 127.18.154.3:44287
I20260807 18:15:59.841271 7258 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.154.3:44287 every 8 connection(s)
I20260807 18:15:59.842669 7076 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/ts-2/data/info.pb
I20260807 18:15:59.852586 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 7076
I20260807 18:15:59.852813 19048 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/ts-2/wal/instance
I20260807 18:15:59.853401 19048 external_mini_cluster.cc:1428] Setting key 3e648b8a7904873260eae2e8b2b4c883
Aug 07 18:15:59 dist-test-slave-gkw7 krb5kdc[6552](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126559, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.3@KRBTEST.COM for kudu/127.18.154.62@KRBTEST.COM
I20260807 18:15:59.865082 7266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:59.845735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.3:38419 (local address 127.18.154.62:39535)
0807 18:15:59.846294 (+ 559us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:59.846299 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:59.847280 (+ 981us) server_negotiation.cc:407] Connection header received
0807 18:15:59.849380 (+ 2100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:59.849385 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:59.849473 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:59.849686 (+ 213us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:15:59.851240 (+ 1554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:59.852087 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:59.854461 (+ 2374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:59.854909 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:59.855055 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:59.858862 (+ 3807us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:15:59.858868 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:15:59.858872 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:15:59.858908 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:15:59.862091 (+ 3183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:15:59.862911 (+ 820us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:15:59.862916 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:15:59.862919 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:15:59.863051 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:15:59.863503 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:15:59.863507 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:15:59.863509 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:15:59.863817 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:15:59.863942 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0807 18:15:59.864616 (+ 674us) server_negotiation.cc:299] Negotiation successful
0807 18:15:59.864806 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":443,"thread_start_us":294,"threads_started":1}
I20260807 18:15:59.866186 7268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:59.846306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:39535 (local address 127.18.154.3:38419)
0807 18:15:59.847024 (+ 718us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:59.847099 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0807 18:15:59.849073 (+ 1974us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:15:59.849880 (+ 807us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:15:59.849888 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:15:59.850463 (+ 575us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:15:59.851060 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:59.851077 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:59.852270 (+ 1193us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:59.852277 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:59.854299 (+ 2022us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:59.854312 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:59.855166 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:59.855174 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:59.855412 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:59.856127 (+ 715us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:15:59.856160 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:15:59.858625 (+ 2465us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:15:59.862225 (+ 3600us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:15:59.862232 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:15:59.862246 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:15:59.862688 (+ 442us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:15:59.863175 (+ 487us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:15:59.863178 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:15:59.863180 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:15:59.863382 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:15:59.863926 (+ 544us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:15:59.863933 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:15:59.864280 (+ 347us) client_negotiation.cc:769] Sending connection context
0807 18:15:59.864511 (+ 231us) client_negotiation.cc:240] Negotiation successful
0807 18:15:59.864934 (+ 423us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":478,"thread_start_us":277,"threads_started":1}
I20260807 18:15:59.867878 7260 heartbeater.cc:347] Connected to a master server at 127.18.154.62:39535
I20260807 18:15:59.868278 7260 heartbeater.cc:464] Registering TS with master...
I20260807 18:15:59.869230 7260 heartbeater.cc:511] Master 127.18.154.62:39535 requested a full tablet report, sending...
I20260807 18:15:59.870890 6618 ts_manager.cc:194] Registered new tserver with Master: 9fc0d7d72d654b3fbed6817c5715dc59 (127.18.154.3:44287)
I20260807 18:15:59.871730 6618 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.154.3@KRBTEST.COM'} at 127.18.154.3:38419
I20260807 18:15:59.882586 19048 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 18:15:59.902534 7266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:59.887698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50038 (local address 127.18.154.62:39535)
0807 18:15:59.887920 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:59.887927 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:59.888383 (+ 456us) server_negotiation.cc:407] Connection header received
0807 18:15:59.888594 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:59.888607 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:59.888671 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:59.888979 (+ 308us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:15:59.890716 (+ 1737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:59.891482 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:59.892921 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:59.893352 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:59.893499 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:59.896018 (+ 2519us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:15:59.896024 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:15:59.896027 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:15:59.896064 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:15:59.898893 (+ 2829us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:15:59.899739 (+ 846us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:15:59.899752 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:15:59.899754 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:15:59.899883 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:15:59.900284 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:15:59.900290 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:15:59.900295 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:15:59.900858 (+ 563us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:15:59.901395 (+ 537us) server_negotiation.cc:1035] Waiting for connection context
0807 18:15:59.901902 (+ 507us) server_negotiation.cc:299] Negotiation successful
0807 18:15:59.902172 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260807 18:15:59.907101 6618 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50038:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 18:15:59.914458 6618 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:15:59.941864 7292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:59.930197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60874 (local address 127.18.154.2:45587)
0807 18:15:59.933832 (+ 3635us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:59.933844 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:59.933876 (+ 32us) server_negotiation.cc:407] Connection header received
0807 18:15:59.933988 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:59.933999 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:59.934244 (+ 245us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:59.934457 (+ 213us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:15:59.935021 (+ 564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:59.936614 (+ 1593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:59.938868 (+ 2254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:59.940421 (+ 1553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:59.940581 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:59.940827 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0807 18:15:59.941224 (+ 397us) server_negotiation.cc:299] Negotiation successful
0807 18:15:59.941412 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3465,"thread_start_us":3297,"threads_started":1}
I20260807 18:15:59.942023 7287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:59.928638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.1:44921 (local address 127.0.0.1:58320)
0807 18:15:59.929487 (+ 849us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:59.929579 (+ 92us) client_negotiation.cc:174] Beginning negotiation
0807 18:15:59.929817 (+ 238us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:15:59.932478 (+ 2661us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:15:59.932491 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:15:59.932541 (+ 50us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:15:59.933186 (+ 645us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:59.933206 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:59.935646 (+ 2440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:59.935652 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:59.938695 (+ 3043us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:59.938712 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:59.940879 (+ 2167us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:59.940887 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:59.941081 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:59.941138 (+ 57us) client_negotiation.cc:769] Sending connection context
0807 18:15:59.941223 (+ 85us) client_negotiation.cc:240] Negotiation successful
0807 18:15:59.941405 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":538,"mutex_wait_us":28,"thread_start_us":329,"threads_started":1}
I20260807 18:15:59.942049 7289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:59.929456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.2:45587 (local address 127.0.0.1:60874)
0807 18:15:59.930549 (+ 1093us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:59.930585 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0807 18:15:59.930734 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:15:59.934441 (+ 3707us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:15:59.934446 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:15:59.934473 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:15:59.934855 (+ 382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:59.934865 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:59.936759 (+ 1894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:59.936772 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:59.938689 (+ 1917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:59.938703 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:59.940640 (+ 1937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:59.940647 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:59.940879 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:59.940947 (+ 68us) client_negotiation.cc:769] Sending connection context
0807 18:15:59.941119 (+ 172us) client_negotiation.cc:240] Negotiation successful
0807 18:15:59.941397 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":961,"mutex_wait_us":74,"thread_start_us":985,"threads_started":1}
I20260807 18:15:59.942512 7288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:59.928849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58320 (local address 127.18.154.1:44921)
0807 18:15:59.931552 (+ 2703us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:59.931579 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:59.931620 (+ 41us) server_negotiation.cc:407] Connection header received
0807 18:15:59.931760 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:59.931773 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:59.932120 (+ 347us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:59.933497 (+ 1377us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:15:59.933826 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:59.935462 (+ 1636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:59.938924 (+ 3462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:59.940605 (+ 1681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:59.941608 (+ 1003us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:59.941881 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0807 18:15:59.941966 (+ 85us) server_negotiation.cc:299] Negotiation successful
0807 18:15:59.942151 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2445,"thread_start_us":2226,"threads_started":1}
I20260807 18:15:59.943234 7293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:59.930929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54492 (local address 127.18.154.3:44287)
0807 18:15:59.931926 (+ 997us) server_negotiation.cc:206] Beginning negotiation
0807 18:15:59.931939 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 18:15:59.932739 (+ 800us) server_negotiation.cc:407] Connection header received
0807 18:15:59.932849 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:15:59.932857 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:15:59.933125 (+ 268us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:15:59.933336 (+ 211us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:15:59.934389 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:59.936185 (+ 1796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:59.939129 (+ 2944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:15:59.940671 (+ 1542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:15:59.940806 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:59.941082 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0807 18:15:59.942648 (+ 1566us) server_negotiation.cc:299] Negotiation successful
0807 18:15:59.942813 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":749,"thread_start_us":562,"threads_started":1}
I20260807 18:15:59.943300 7291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:15:59.930128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.3:44287 (local address 127.0.0.1:54492)
0807 18:15:59.932424 (+ 2296us) negotiation.cc:107] Waiting for socket to connect
0807 18:15:59.932444 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 18:15:59.932585 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:15:59.933723 (+ 1138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:15:59.933728 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:15:59.933755 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:15:59.934177 (+ 422us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:59.934191 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:59.936378 (+ 2187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:59.936386 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:59.938923 (+ 2537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:15:59.938937 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:15:59.941012 (+ 2075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:15:59.941018 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:15:59.941228 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:15:59.941260 (+ 32us) client_negotiation.cc:769] Sending connection context
0807 18:15:59.941354 (+ 94us) client_negotiation.cc:240] Negotiation successful
0807 18:15:59.941473 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2166,"mutex_wait_us":3,"thread_start_us":2013,"threads_started":1}
I20260807 18:15:59.946888 6987 tablet_service.cc:1467] Processing CreateTablet for tablet 2841e3dddc06443b90ac0d5e623bdd4e (DEFAULT_TABLE table=test-table [id=a9a2859f711c43dd9917da54e537f167]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:15:59.947077 6793 tablet_service.cc:1467] Processing CreateTablet for tablet 2841e3dddc06443b90ac0d5e623bdd4e (DEFAULT_TABLE table=test-table [id=a9a2859f711c43dd9917da54e537f167]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:15:59.947240 7169 tablet_service.cc:1467] Processing CreateTablet for tablet 2841e3dddc06443b90ac0d5e623bdd4e (DEFAULT_TABLE table=test-table [id=a9a2859f711c43dd9917da54e537f167]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:15:59.948239 6987 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2841e3dddc06443b90ac0d5e623bdd4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:15:59.948716 6793 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2841e3dddc06443b90ac0d5e623bdd4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:15:59.948719 7169 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2841e3dddc06443b90ac0d5e623bdd4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:15:59.960155 7294 tablet_bootstrap.cc:492] T 2841e3dddc06443b90ac0d5e623bdd4e P 20725a0434e54ab9b7d73c8733499c30: Bootstrap starting.
I20260807 18:15:59.964640 7294 tablet_bootstrap.cc:654] T 2841e3dddc06443b90ac0d5e623bdd4e P 20725a0434e54ab9b7d73c8733499c30: Neither blocks nor log segments found. Creating new log.
I20260807 18:15:59.965045 7295 tablet_bootstrap.cc:492] T 2841e3dddc06443b90ac0d5e623bdd4e P 70281fa69b7949aead7a99ec552fa256: Bootstrap starting.
I20260807 18:15:59.966284 7294 log.cc:824] T 2841e3dddc06443b90ac0d5e623bdd4e P 20725a0434e54ab9b7d73c8733499c30: Log is configured to *not* fsync() on all Append() calls
I20260807 18:15:59.967908 7297 tablet_bootstrap.cc:492] T 2841e3dddc06443b90ac0d5e623bdd4e P 9fc0d7d72d654b3fbed6817c5715dc59: Bootstrap starting.
I20260807 18:15:59.969027 7295 tablet_bootstrap.cc:654] T 2841e3dddc06443b90ac0d5e623bdd4e P 70281fa69b7949aead7a99ec552fa256: Neither blocks nor log segments found. Creating new log.
I20260807 18:15:59.970434 7294 tablet_bootstrap.cc:492] T 2841e3dddc06443b90ac0d5e623bdd4e P 20725a0434e54ab9b7d73c8733499c30: No bootstrap required, opened a new log
I20260807 18:15:59.970444 7295 log.cc:824] T 2841e3dddc06443b90ac0d5e623bdd4e P 70281fa69b7949aead7a99ec552fa256: Log is configured to *not* fsync() on all Append() calls
I20260807 18:15:59.970743 7294 ts_tablet_manager.cc:1397] T 2841e3dddc06443b90ac0d5e623bdd4e P 20725a0434e54ab9b7d73c8733499c30: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.005s
I20260807 18:15:59.971658 7297 tablet_bootstrap.cc:654] T 2841e3dddc06443b90ac0d5e623bdd4e P 9fc0d7d72d654b3fbed6817c5715dc59: Neither blocks nor log segments found. Creating new log.
I20260807 18:15:59.973040 7297 log.cc:824] T 2841e3dddc06443b90ac0d5e623bdd4e P 9fc0d7d72d654b3fbed6817c5715dc59: Log is configured to *not* fsync() on all Append() calls
I20260807 18:15:59.974129 7295 tablet_bootstrap.cc:492] T 2841e3dddc06443b90ac0d5e623bdd4e P 70281fa69b7949aead7a99ec552fa256: No bootstrap required, opened a new log
I20260807 18:15:59.974450 7295 ts_tablet_manager.cc:1397] T 2841e3dddc06443b90ac0d5e623bdd4e P 70281fa69b7949aead7a99ec552fa256: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.002s
I20260807 18:15:59.976492 7297 tablet_bootstrap.cc:492] T 2841e3dddc06443b90ac0d5e623bdd4e P 9fc0d7d72d654b3fbed6817c5715dc59: No bootstrap required, opened a new log
I20260807 18:15:59.976773 7297 ts_tablet_manager.cc:1397] T 2841e3dddc06443b90ac0d5e623bdd4e P 9fc0d7d72d654b3fbed6817c5715dc59: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.006s
I20260807 18:15:59.980191 7294 raft_consensus.cc:348] T 2841e3dddc06443b90ac0d5e623bdd4e P 20725a0434e54ab9b7d73c8733499c30 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70281fa69b7949aead7a99ec552fa256" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 44921 } } peers { permanent_uuid: "9fc0d7d72d654b3fbed6817c5715dc59" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 44287 } } peers { permanent_uuid: "20725a0434e54ab9b7d73c8733499c30" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 45587 } }
I20260807 18:15:59.980605 7294 raft_consensus.cc:374] T 2841e3dddc06443b90ac0d5e623bdd4e P 20725a0434e54ab9b7d73c8733499c30 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:15:59.980748 7294 raft_consensus.cc:729] T 2841e3dddc06443b90ac0d5e623bdd4e P 20725a0434e54ab9b7d73c8733499c30 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20725a0434e54ab9b7d73c8733499c30, State: Initialized, Role: FOLLOWER
I20260807 18:15:59.981410 7294 consensus_queue.cc:260] T 2841e3dddc06443b90ac0d5e623bdd4e P 20725a0434e54ab9b7d73c8733499c30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70281fa69b7949aead7a99ec552fa256" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 44921 } } peers { permanent_uuid: "9fc0d7d72d654b3fbed6817c5715dc59" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 44287 } } peers { permanent_uuid: "20725a0434e54ab9b7d73c8733499c30" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 45587 } }
I20260807 18:15:59.983503 7294 ts_tablet_manager.cc:1428] T 2841e3dddc06443b90ac0d5e623bdd4e P 20725a0434e54ab9b7d73c8733499c30: Time spent starting tablet: real 0.013s user 0.003s sys 0.009s
I20260807 18:15:59.984278 7069 heartbeater.cc:503] Master 127.18.154.62:39535 was elected leader, sending a full tablet report...
I20260807 18:15:59.985927 7295 raft_consensus.cc:348] T 2841e3dddc06443b90ac0d5e623bdd4e P 70281fa69b7949aead7a99ec552fa256 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70281fa69b7949aead7a99ec552fa256" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 44921 } } peers { permanent_uuid: "9fc0d7d72d654b3fbed6817c5715dc59" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 44287 } } peers { permanent_uuid: "20725a0434e54ab9b7d73c8733499c30" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 45587 } }
I20260807 18:15:59.986393 7295 raft_consensus.cc:374] T 2841e3dddc06443b90ac0d5e623bdd4e P 70281fa69b7949aead7a99ec552fa256 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:15:59.986490 7295 raft_consensus.cc:729] T 2841e3dddc06443b90ac0d5e623bdd4e P 70281fa69b7949aead7a99ec552fa256 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 70281fa69b7949aead7a99ec552fa256, State: Initialized, Role: FOLLOWER
I20260807 18:15:59.987136 7295 consensus_queue.cc:260] T 2841e3dddc06443b90ac0d5e623bdd4e P 70281fa69b7949aead7a99ec552fa256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70281fa69b7949aead7a99ec552fa256" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 44921 } } peers { permanent_uuid: "9fc0d7d72d654b3fbed6817c5715dc59" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 44287 } } peers { permanent_uuid: "20725a0434e54ab9b7d73c8733499c30" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 45587 } }
I20260807 18:15:59.987637 7297 raft_consensus.cc:348] T 2841e3dddc06443b90ac0d5e623bdd4e P 9fc0d7d72d654b3fbed6817c5715dc59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70281fa69b7949aead7a99ec552fa256" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 44921 } } peers { permanent_uuid: "9fc0d7d72d654b3fbed6817c5715dc59" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 44287 } } peers { permanent_uuid: "20725a0434e54ab9b7d73c8733499c30" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 45587 } }
I20260807 18:15:59.988183 7297 raft_consensus.cc:374] T 2841e3dddc06443b90ac0d5e623bdd4e P 9fc0d7d72d654b3fbed6817c5715dc59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:15:59.988312 7297 raft_consensus.cc:729] T 2841e3dddc06443b90ac0d5e623bdd4e P 9fc0d7d72d654b3fbed6817c5715dc59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9fc0d7d72d654b3fbed6817c5715dc59, State: Initialized, Role: FOLLOWER
I20260807 18:15:59.989169 7297 consensus_queue.cc:260] T 2841e3dddc06443b90ac0d5e623bdd4e P 9fc0d7d72d654b3fbed6817c5715dc59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70281fa69b7949aead7a99ec552fa256" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 44921 } } peers { permanent_uuid: "9fc0d7d72d654b3fbed6817c5715dc59" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 44287 } } peers { permanent_uuid: "20725a0434e54ab9b7d73c8733499c30" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 45587 } }
I20260807 18:15:59.990995 7295 ts_tablet_manager.cc:1428] T 2841e3dddc06443b90ac0d5e623bdd4e P 70281fa69b7949aead7a99ec552fa256: Time spent starting tablet: real 0.016s user 0.011s sys 0.005s
I20260807 18:15:59.994349 7260 heartbeater.cc:503] Master 127.18.154.62:39535 was elected leader, sending a full tablet report...
I20260807 18:15:59.994824 7297 ts_tablet_manager.cc:1428] T 2841e3dddc06443b90ac0d5e623bdd4e P 9fc0d7d72d654b3fbed6817c5715dc59: Time spent starting tablet: real 0.018s user 0.017s sys 0.001s
W20260807 18:16:00.036048 7070 tablet.cc:2369] T 2841e3dddc06443b90ac0d5e623bdd4e P 20725a0434e54ab9b7d73c8733499c30: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 18:16:00.094408 7261 tablet.cc:2369] T 2841e3dddc06443b90ac0d5e623bdd4e P 9fc0d7d72d654b3fbed6817c5715dc59: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:16:00.203955 7305 raft_consensus.cc:482] T 2841e3dddc06443b90ac0d5e623bdd4e P 20725a0434e54ab9b7d73c8733499c30 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:16:00.204350 7305 raft_consensus.cc:504] T 2841e3dddc06443b90ac0d5e623bdd4e P 20725a0434e54ab9b7d73c8733499c30 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70281fa69b7949aead7a99ec552fa256" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 44921 } } peers { permanent_uuid: "9fc0d7d72d654b3fbed6817c5715dc59" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 44287 } } peers { permanent_uuid: "20725a0434e54ab9b7d73c8733499c30" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 45587 } }
I20260807 18:16:00.206310 7305 leader_election.cc:288] T 2841e3dddc06443b90ac0d5e623bdd4e P 20725a0434e54ab9b7d73c8733499c30 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 70281fa69b7949aead7a99ec552fa256 (127.18.154.1:44921), 9fc0d7d72d654b3fbed6817c5715dc59 (127.18.154.3:44287)
W20260807 18:16:00.210886 6894 tablet.cc:2369] T 2841e3dddc06443b90ac0d5e623bdd4e P 70281fa69b7949aead7a99ec552fa256: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:16:00.213989 7293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:00.207501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.2:38761 (local address 127.18.154.3:44287)
0807 18:16:00.207741 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0807 18:16:00.207748 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:16:00.208153 (+ 405us) server_negotiation.cc:407] Connection header received
0807 18:16:00.208300 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:16:00.208306 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:16:00.208395 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:16:00.208498 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:16:00.209164 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:16:00.210124 (+ 960us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:16:00.212015 (+ 1891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:16:00.212897 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:16:00.213030 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:16:00.213083 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0807 18:16:00.213575 (+ 492us) server_negotiation.cc:299] Negotiation successful
0807 18:16:00.213736 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260807 18:16:00.214097 7288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:00.206915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.2:41287 (local address 127.18.154.1:44921)
0807 18:16:00.207195 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0807 18:16:00.207203 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:16:00.207435 (+ 232us) server_negotiation.cc:407] Connection header received
0807 18:16:00.207753 (+ 318us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:16:00.207762 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:16:00.207846 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:16:00.207899 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:16:00.208787 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:16:00.209792 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:16:00.212247 (+ 2455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:16:00.213111 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:16:00.213275 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:16:00.213347 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0807 18:16:00.213730 (+ 383us) server_negotiation.cc:299] Negotiation successful
0807 18:16:00.213866 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260807 18:16:00.214105 7312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:00.207355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.3:44287 (local address 127.18.154.2:38761)
0807 18:16:00.207996 (+ 641us) negotiation.cc:107] Waiting for socket to connect
0807 18:16:00.208011 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 18:16:00.208169 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:16:00.208510 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:16:00.208513 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:16:00.208535 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:16:00.208974 (+ 439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:16:00.208987 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:16:00.210264 (+ 1277us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:16:00.210268 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:16:00.211862 (+ 1594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:16:00.211875 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:16:00.213125 (+ 1250us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:16:00.213143 (+ 18us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:16:00.213353 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:16:00.213396 (+ 43us) client_negotiation.cc:769] Sending connection context
0807 18:16:00.213474 (+ 78us) client_negotiation.cc:240] Negotiation successful
0807 18:16:00.213635 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":544,"spinlock_wait_cycles":13056,"thread_start_us":323,"threads_started":1}
I20260807 18:16:00.214790 7311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:00.206545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.1:44921 (local address 127.18.154.2:41287)
0807 18:16:00.207286 (+ 741us) negotiation.cc:107] Waiting for socket to connect
0807 18:16:00.207308 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 18:16:00.207515 (+ 207us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:16:00.208096 (+ 581us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:16:00.208100 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:16:00.208167 (+ 67us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:16:00.208607 (+ 440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:16:00.208623 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:16:00.209920 (+ 1297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:16:00.209929 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:16:00.212083 (+ 2154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:16:00.212098 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:16:00.213327 (+ 1229us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:16:00.213331 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:16:00.213494 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:16:00.213526 (+ 32us) client_negotiation.cc:769] Sending connection context
0807 18:16:00.214319 (+ 793us) client_negotiation.cc:240] Negotiation successful
0807 18:16:00.214455 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":602,"spinlock_wait_cycles":896,"thread_start_us":361,"threads_started":1}
I20260807 18:16:00.215302 7197 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2841e3dddc06443b90ac0d5e623bdd4e" candidate_uuid: "20725a0434e54ab9b7d73c8733499c30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9fc0d7d72d654b3fbed6817c5715dc59" is_pre_election: true
I20260807 18:16:00.215914 7197 raft_consensus.cc:2454] T 2841e3dddc06443b90ac0d5e623bdd4e P 9fc0d7d72d654b3fbed6817c5715dc59 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20725a0434e54ab9b7d73c8733499c30 in term 0.
I20260807 18:16:00.215894 6825 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2841e3dddc06443b90ac0d5e623bdd4e" candidate_uuid: "20725a0434e54ab9b7d73c8733499c30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "70281fa69b7949aead7a99ec552fa256" is_pre_election: true
I20260807 18:16:00.216506 6825 raft_consensus.cc:2454] T 2841e3dddc06443b90ac0d5e623bdd4e P 70281fa69b7949aead7a99ec552fa256 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20725a0434e54ab9b7d73c8733499c30 in term 0.
I20260807 18:16:00.216722 6924 leader_election.cc:302] T 2841e3dddc06443b90ac0d5e623bdd4e P 20725a0434e54ab9b7d73c8733499c30 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20725a0434e54ab9b7d73c8733499c30, 9fc0d7d72d654b3fbed6817c5715dc59; no voters:
I20260807 18:16:00.217144 7305 raft_consensus.cc:2781] T 2841e3dddc06443b90ac0d5e623bdd4e P 20725a0434e54ab9b7d73c8733499c30 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:16:00.217304 7305 raft_consensus.cc:482] T 2841e3dddc06443b90ac0d5e623bdd4e P 20725a0434e54ab9b7d73c8733499c30 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:16:00.217418 7305 raft_consensus.cc:3037] T 2841e3dddc06443b90ac0d5e623bdd4e P 20725a0434e54ab9b7d73c8733499c30 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:16:00.219669 7305 raft_consensus.cc:504] T 2841e3dddc06443b90ac0d5e623bdd4e P 20725a0434e54ab9b7d73c8733499c30 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70281fa69b7949aead7a99ec552fa256" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 44921 } } peers { permanent_uuid: "9fc0d7d72d654b3fbed6817c5715dc59" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 44287 } } peers { permanent_uuid: "20725a0434e54ab9b7d73c8733499c30" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 45587 } }
I20260807 18:16:00.220275 7305 leader_election.cc:288] T 2841e3dddc06443b90ac0d5e623bdd4e P 20725a0434e54ab9b7d73c8733499c30 [CANDIDATE]: Term 1 election: Requested vote from peers 70281fa69b7949aead7a99ec552fa256 (127.18.154.1:44921), 9fc0d7d72d654b3fbed6817c5715dc59 (127.18.154.3:44287)
I20260807 18:16:00.220682 6825 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2841e3dddc06443b90ac0d5e623bdd4e" candidate_uuid: "20725a0434e54ab9b7d73c8733499c30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "70281fa69b7949aead7a99ec552fa256"
I20260807 18:16:00.220827 7197 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2841e3dddc06443b90ac0d5e623bdd4e" candidate_uuid: "20725a0434e54ab9b7d73c8733499c30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9fc0d7d72d654b3fbed6817c5715dc59"
I20260807 18:16:00.220878 6825 raft_consensus.cc:3037] T 2841e3dddc06443b90ac0d5e623bdd4e P 70281fa69b7949aead7a99ec552fa256 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:16:00.221053 7197 raft_consensus.cc:3037] T 2841e3dddc06443b90ac0d5e623bdd4e P 9fc0d7d72d654b3fbed6817c5715dc59 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:16:00.223150 6825 raft_consensus.cc:2454] T 2841e3dddc06443b90ac0d5e623bdd4e P 70281fa69b7949aead7a99ec552fa256 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20725a0434e54ab9b7d73c8733499c30 in term 1.
I20260807 18:16:00.223443 7197 raft_consensus.cc:2454] T 2841e3dddc06443b90ac0d5e623bdd4e P 9fc0d7d72d654b3fbed6817c5715dc59 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20725a0434e54ab9b7d73c8733499c30 in term 1.
I20260807 18:16:00.223626 6924 leader_election.cc:302] T 2841e3dddc06443b90ac0d5e623bdd4e P 20725a0434e54ab9b7d73c8733499c30 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20725a0434e54ab9b7d73c8733499c30, 70281fa69b7949aead7a99ec552fa256; no voters:
I20260807 18:16:00.223940 7305 raft_consensus.cc:2781] T 2841e3dddc06443b90ac0d5e623bdd4e P 20725a0434e54ab9b7d73c8733499c30 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:16:00.224432 7305 raft_consensus.cc:686] T 2841e3dddc06443b90ac0d5e623bdd4e P 20725a0434e54ab9b7d73c8733499c30 [term 1 LEADER]: Becoming Leader. State: Replica: 20725a0434e54ab9b7d73c8733499c30, State: Running, Role: LEADER
I20260807 18:16:00.224931 7305 consensus_queue.cc:237] T 2841e3dddc06443b90ac0d5e623bdd4e P 20725a0434e54ab9b7d73c8733499c30 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70281fa69b7949aead7a99ec552fa256" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 44921 } } peers { permanent_uuid: "9fc0d7d72d654b3fbed6817c5715dc59" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 44287 } } peers { permanent_uuid: "20725a0434e54ab9b7d73c8733499c30" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 45587 } }
I20260807 18:16:00.230000 6618 catalog_manager.cc:5495] T 2841e3dddc06443b90ac0d5e623bdd4e P 20725a0434e54ab9b7d73c8733499c30 reported cstate change: term changed from 0 to 1, leader changed from <none> to 20725a0434e54ab9b7d73c8733499c30 (127.18.154.2). New cstate: current_term: 1 leader_uuid: "20725a0434e54ab9b7d73c8733499c30" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70281fa69b7949aead7a99ec552fa256" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 44921 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9fc0d7d72d654b3fbed6817c5715dc59" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 44287 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20725a0434e54ab9b7d73c8733499c30" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 45587 } health_report { overall_health: HEALTHY } } }
I20260807 18:16:00.312598 7292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:00.306952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60888 (local address 127.18.154.2:45587)
0807 18:16:00.307195 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0807 18:16:00.307200 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:16:00.307218 (+ 18us) server_negotiation.cc:407] Connection header received
0807 18:16:00.307319 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:16:00.307323 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:16:00.307401 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:16:00.307529 (+ 128us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:16:00.307897 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:16:00.308674 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:16:00.310279 (+ 1605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:16:00.310702 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:16:00.310810 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:16:00.311377 (+ 567us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:16:00.311919 (+ 542us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:16:00.312042 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0807 18:16:00.312186 (+ 144us) server_negotiation.cc:299] Negotiation successful
0807 18:16:00.312260 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260807 18:16:00.321278 7197 raft_consensus.cc:1261] T 2841e3dddc06443b90ac0d5e623bdd4e P 9fc0d7d72d654b3fbed6817c5715dc59 [term 1 FOLLOWER]: Refusing update from remote peer 20725a0434e54ab9b7d73c8733499c30: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:16:00.321348 6825 raft_consensus.cc:1261] T 2841e3dddc06443b90ac0d5e623bdd4e P 70281fa69b7949aead7a99ec552fa256 [term 1 FOLLOWER]: Refusing update from remote peer 20725a0434e54ab9b7d73c8733499c30: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:16:00.322386 7313 consensus_queue.cc:1035] T 2841e3dddc06443b90ac0d5e623bdd4e P 20725a0434e54ab9b7d73c8733499c30 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9fc0d7d72d654b3fbed6817c5715dc59" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 44287 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:16:00.322829 7305 consensus_queue.cc:1035] T 2841e3dddc06443b90ac0d5e623bdd4e P 20725a0434e54ab9b7d73c8733499c30 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70281fa69b7949aead7a99ec552fa256" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 44921 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:16:00.334826 7317 mvcc.cc:205] Tried to move back new op lower bound from 7315974391065493504 to 7315974390685859840. Current Snapshot: MvccSnapshot[applied={T|T < 7315974391065493504}]
I20260807 18:16:00.336274 7320 mvcc.cc:205] Tried to move back new op lower bound from 7315974391065493504 to 7315974390685859840. Current Snapshot: MvccSnapshot[applied={T|T < 7315974391065493504}]
I20260807 18:16:00.349291 6618 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50038:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:16:00.349617 6618 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50038:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:16:00.353497 6618 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d800f8a66628417d808b2e4b0961b702: Sending DeleteTablet for 3 replicas of tablet 2841e3dddc06443b90ac0d5e623bdd4e
I20260807 18:16:00.354938 6987 tablet_service.cc:1514] Processing DeleteTablet for tablet 2841e3dddc06443b90ac0d5e623bdd4e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:16:00 UTC) from {username='kudu', principal='kudu/127.18.154.62@KRBTEST.COM'} at 127.0.0.1:60874
I20260807 18:16:00.354897 6793 tablet_service.cc:1514] Processing DeleteTablet for tablet 2841e3dddc06443b90ac0d5e623bdd4e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:16:00 UTC) from {username='kudu', principal='kudu/127.18.154.62@KRBTEST.COM'} at 127.0.0.1:58320
I20260807 18:16:00.354954 7169 tablet_service.cc:1514] Processing DeleteTablet for tablet 2841e3dddc06443b90ac0d5e623bdd4e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:16:00 UTC) from {username='kudu', principal='kudu/127.18.154.62@KRBTEST.COM'} at 127.0.0.1:54492
I20260807 18:16:00.355414 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 6703
I20260807 18:16:00.374831 7329 tablet_replica.cc:330] T 2841e3dddc06443b90ac0d5e623bdd4e P 9fc0d7d72d654b3fbed6817c5715dc59: stopping tablet replica
I20260807 18:16:00.375437 7329 raft_consensus.cc:2229] T 2841e3dddc06443b90ac0d5e623bdd4e P 9fc0d7d72d654b3fbed6817c5715dc59 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:16:00.375953 7329 raft_consensus.cc:2258] T 2841e3dddc06443b90ac0d5e623bdd4e P 9fc0d7d72d654b3fbed6817c5715dc59 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:16:00.376650 7328 tablet_replica.cc:330] T 2841e3dddc06443b90ac0d5e623bdd4e P 20725a0434e54ab9b7d73c8733499c30: stopping tablet replica
I20260807 18:16:00.377048 7328 raft_consensus.cc:2229] T 2841e3dddc06443b90ac0d5e623bdd4e P 20725a0434e54ab9b7d73c8733499c30 [term 1 LEADER]: Raft consensus shutting down.
I20260807 18:16:00.377749 7328 raft_consensus.cc:2258] T 2841e3dddc06443b90ac0d5e623bdd4e P 20725a0434e54ab9b7d73c8733499c30 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:16:00.380345 7329 ts_tablet_manager.cc:1905] T 2841e3dddc06443b90ac0d5e623bdd4e P 9fc0d7d72d654b3fbed6817c5715dc59: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:16:00.381001 7327 tablet_replica.cc:330] T 2841e3dddc06443b90ac0d5e623bdd4e P 70281fa69b7949aead7a99ec552fa256: stopping tablet replica
I20260807 18:16:00.381404 7328 ts_tablet_manager.cc:1905] T 2841e3dddc06443b90ac0d5e623bdd4e P 20725a0434e54ab9b7d73c8733499c30: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:16:00.381430 7327 raft_consensus.cc:2229] T 2841e3dddc06443b90ac0d5e623bdd4e P 70281fa69b7949aead7a99ec552fa256 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:16:00.386250 7327 raft_consensus.cc:2258] T 2841e3dddc06443b90ac0d5e623bdd4e P 70281fa69b7949aead7a99ec552fa256 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:16:00.388540 7327 ts_tablet_manager.cc:1905] T 2841e3dddc06443b90ac0d5e623bdd4e P 70281fa69b7949aead7a99ec552fa256: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:16:00.389108 7328 ts_tablet_manager.cc:1918] T 2841e3dddc06443b90ac0d5e623bdd4e P 20725a0434e54ab9b7d73c8733499c30: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:16:00.389109 7329 ts_tablet_manager.cc:1918] T 2841e3dddc06443b90ac0d5e623bdd4e P 9fc0d7d72d654b3fbed6817c5715dc59: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:16:00.389281 7329 log.cc:1196] T 2841e3dddc06443b90ac0d5e623bdd4e P 9fc0d7d72d654b3fbed6817c5715dc59: Deleting WAL directory at /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/ts-2/wal/wals/2841e3dddc06443b90ac0d5e623bdd4e
I20260807 18:16:00.389286 7328 log.cc:1196] T 2841e3dddc06443b90ac0d5e623bdd4e P 20725a0434e54ab9b7d73c8733499c30: Deleting WAL directory at /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/ts-1/wal/wals/2841e3dddc06443b90ac0d5e623bdd4e
I20260807 18:16:00.389887 7329 ts_tablet_manager.cc:1939] T 2841e3dddc06443b90ac0d5e623bdd4e P 9fc0d7d72d654b3fbed6817c5715dc59: Deleting consensus metadata
I20260807 18:16:00.390676 7328 ts_tablet_manager.cc:1939] T 2841e3dddc06443b90ac0d5e623bdd4e P 20725a0434e54ab9b7d73c8733499c30: Deleting consensus metadata
I20260807 18:16:00.391970 6603 catalog_manager.cc:4841] TS 9fc0d7d72d654b3fbed6817c5715dc59 (127.18.154.3:44287): tablet 2841e3dddc06443b90ac0d5e623bdd4e (table test-table [id=a9a2859f711c43dd9917da54e537f167]) successfully deleted
I20260807 18:16:00.392282 6599 catalog_manager.cc:4841] TS 20725a0434e54ab9b7d73c8733499c30 (127.18.154.2:45587): tablet 2841e3dddc06443b90ac0d5e623bdd4e (table test-table [id=a9a2859f711c43dd9917da54e537f167]) successfully deleted
Aug 07 18:16:00 dist-test-slave-gkw7 krb5kdc[6552](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126557, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.154.1@KRBTEST.COM
I20260807 18:16:00.396520 7327 ts_tablet_manager.cc:1918] T 2841e3dddc06443b90ac0d5e623bdd4e P 70281fa69b7949aead7a99ec552fa256: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:16:00.396698 7327 log.cc:1196] T 2841e3dddc06443b90ac0d5e623bdd4e P 70281fa69b7949aead7a99ec552fa256: Deleting WAL directory at /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/ts-0/wal/wals/2841e3dddc06443b90ac0d5e623bdd4e
I20260807 18:16:00.397289 7327 ts_tablet_manager.cc:1939] T 2841e3dddc06443b90ac0d5e623bdd4e P 70281fa69b7949aead7a99ec552fa256: Deleting consensus metadata
I20260807 18:16:00.399298 6603 catalog_manager.cc:4841] TS 70281fa69b7949aead7a99ec552fa256 (127.18.154.1:44921): tablet 2841e3dddc06443b90ac0d5e623bdd4e (table test-table [id=a9a2859f711c43dd9917da54e537f167]) successfully deleted
I20260807 18:16:00.403184 7288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:00.357056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58328 (local address 127.18.154.1:44921)
0807 18:16:00.357308 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0807 18:16:00.357317 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:16:00.366218 (+ 8901us) server_negotiation.cc:407] Connection header received
0807 18:16:00.366831 (+ 613us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:16:00.366839 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:16:00.366971 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:16:00.367067 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:16:00.369877 (+ 2810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:16:00.371547 (+ 1670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:16:00.379041 (+ 7494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:16:00.379729 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:16:00.379887 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:16:00.395893 (+ 16006us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:16:00.395909 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:16:00.395927 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:16:00.395970 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:16:00.399950 (+ 3980us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:16:00.400855 (+ 905us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:16:00.400862 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:16:00.400864 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:16:00.400994 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:16:00.401407 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:16:00.401410 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:16:00.401412 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:16:00.401844 (+ 432us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:16:00.402071 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0807 18:16:00.402540 (+ 469us) server_negotiation.cc:299] Negotiation successful
0807 18:16:00.402722 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260807 18:16:00.403940 6888 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:16:00.500564 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 6703
I20260807 18:16:00.529331 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 6899
Aug 07 18:16:00 dist-test-slave-gkw7 krb5kdc[6552](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126557, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.154.2@KRBTEST.COM
I20260807 18:16:00.553236 7292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:00.530264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60892 (local address 127.18.154.2:45587)
0807 18:16:00.530474 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0807 18:16:00.530480 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:16:00.530503 (+ 23us) server_negotiation.cc:407] Connection header received
0807 18:16:00.530674 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:16:00.530680 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:16:00.530763 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:16:00.530888 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:16:00.532248 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:16:00.533495 (+ 1247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:16:00.535034 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:16:00.535749 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:16:00.535877 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:16:00.545520 (+ 9643us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:16:00.545576 (+ 56us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:16:00.545600 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:16:00.545666 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:16:00.550016 (+ 4350us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:16:00.550850 (+ 834us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:16:00.550860 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:16:00.550864 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:16:00.551091 (+ 227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:16:00.551530 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:16:00.551535 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:16:00.551538 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:16:00.552014 (+ 476us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:16:00.552398 (+ 384us) server_negotiation.cc:1035] Waiting for connection context
0807 18:16:00.552558 (+ 160us) server_negotiation.cc:299] Negotiation successful
0807 18:16:00.552823 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260807 18:16:00.554111 7061 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:16:00.670472 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 6899
I20260807 18:16:00.695590 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 7076
Aug 07 18:16:00 dist-test-slave-gkw7 krb5kdc[6552](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126557, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.154.3@KRBTEST.COM
I20260807 18:16:00.715204 7293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:00.696402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54504 (local address 127.18.154.3:44287)
0807 18:16:00.696570 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0807 18:16:00.696575 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:16:00.696593 (+ 18us) server_negotiation.cc:407] Connection header received
0807 18:16:00.696744 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:16:00.696749 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:16:00.696815 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:16:00.696912 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:16:00.698272 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:16:00.699006 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:16:00.700291 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:16:00.700673 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:16:00.700770 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:16:00.708910 (+ 8140us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:16:00.708929 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:16:00.708948 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:16:00.708995 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:16:00.712030 (+ 3035us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:16:00.712921 (+ 891us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:16:00.712930 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:16:00.712933 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:16:00.713112 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:16:00.713674 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:16:00.713679 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:16:00.713681 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:16:00.714094 (+ 413us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:16:00.714354 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0807 18:16:00.714752 (+ 398us) server_negotiation.cc:299] Negotiation successful
0807 18:16:00.714940 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 18:16:00.716073 7253 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:16:00.808461 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 7076
I20260807 18:16:00.827962 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 6574
I20260807 18:16:00.829348 6662 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:16:00.943320 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 6574
I20260807 18:16:00.970605 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.154.62:39535
--webserver_interface=127.18.154.62
--webserver_port=36013
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.62.keytab
--principal=kudu/127.18.154.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.154.20:44041
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.154.62:39535
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:16:01.332593 7339 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:16:01.332909 7339 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:16:01.333004 7339 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:16:01.340421 7339 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:16:01.340543 7339 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:16:01.340591 7339 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:16:01.340651 7339 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:16:01.340694 7339 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:16:01.350919 7339 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.154.20:44041
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.18.154.62:39535
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.62.keytab
--principal=kudu/127.18.154.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.154.62:39535
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.154.62
--webserver_port=36013
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:16:01.352286 7339 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:16:01.353670 7339 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:16:01.381310 7354 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:16:01.381381 7351 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:16:01.381402 7350 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:16:01.382323 7339 server_base.cc:1034] running on GCE node
I20260807 18:16:01.383342 7339 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:16:01.385226 7339 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:16:01.386441 7339 hybrid_clock.cc:648] HybridClock initialized: now 1786126561386391 us; error 35 us; skew 500 ppm
I20260807 18:16:01.386840 7339 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:16:01 dist-test-slave-gkw7 krb5kdc[6552](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126561, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:16:01.390985 7339 init.cc:377] Logged in from keytab as kudu/127.18.154.62@KRBTEST.COM (short username kudu)
I20260807 18:16:01.392867 7339 webserver.cc:460] Webserver started at http://127.18.154.62:36013/ using document root <none> and password file <none>
I20260807 18:16:01.393806 7339 fs_manager.cc:359] Metadata directory not provided
I20260807 18:16:01.393972 7339 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:16:01.400615 7339 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.002s
I20260807 18:16:01.404057 7361 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:16:01.405615 7339 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260807 18:16:01.405817 7339 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/master-0/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/master-0/wal
uuid: "d800f8a66628417d808b2e4b0961b702"
format_stamp: "Formatted at 2026-08-07 18:15:57 on dist-test-slave-gkw7"
server_key: "9b0188e763314f93735051427425d348"
server_key_iv: "9e7ec036ea217b2c0de08e7ac782e263"
server_key_version: "encryptionkey@0"
I20260807 18:16:01.406971 7339 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:16:01.424371 7339 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:16:01.429211 7339 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:16:01.429595 7339 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:16:01.449796 7339 rpc_server.cc:307] RPC server started. Bound to: 127.18.154.62:39535
I20260807 18:16:01.449805 7433 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.154.62:39535 every 8 connection(s)
I20260807 18:16:01.451506 7339 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
I20260807 18:16:01.459158 7434 sys_catalog.cc:263] Verifying existing consensus state
I20260807 18:16:01.460839 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 7339
I20260807 18:16:01.461814 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.154.1:44921
--local_ip_for_outbound_sockets=127.18.154.1
--tserver_master_addrs=127.18.154.62:39535
--webserver_port=36253
--webserver_interface=127.18.154.1
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.1.keytab
--principal=kudu/127.18.154.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.154.20:44041
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 18:16:01.462911 7434 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d800f8a66628417d808b2e4b0961b702: Bootstrap starting.
I20260807 18:16:01.484002 7434 log.cc:824] T 00000000000000000000000000000000 P d800f8a66628417d808b2e4b0961b702: Log is configured to *not* fsync() on all Append() calls
I20260807 18:16:01.496793 7434 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d800f8a66628417d808b2e4b0961b702: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 18:16:01.497366 7434 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d800f8a66628417d808b2e4b0961b702: Bootstrap complete.
I20260807 18:16:01.506960 7434 raft_consensus.cc:348] T 00000000000000000000000000000000 P d800f8a66628417d808b2e4b0961b702 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d800f8a66628417d808b2e4b0961b702" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 39535 } }
I20260807 18:16:01.507846 7434 raft_consensus.cc:729] T 00000000000000000000000000000000 P d800f8a66628417d808b2e4b0961b702 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d800f8a66628417d808b2e4b0961b702, State: Initialized, Role: FOLLOWER
I20260807 18:16:01.508529 7434 consensus_queue.cc:260] T 00000000000000000000000000000000 P d800f8a66628417d808b2e4b0961b702 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d800f8a66628417d808b2e4b0961b702" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 39535 } }
I20260807 18:16:01.508793 7434 raft_consensus.cc:388] T 00000000000000000000000000000000 P d800f8a66628417d808b2e4b0961b702 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:16:01.508931 7434 raft_consensus.cc:482] T 00000000000000000000000000000000 P d800f8a66628417d808b2e4b0961b702 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:16:01.509114 7434 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d800f8a66628417d808b2e4b0961b702 [term 1 FOLLOWER]: Advancing to term 2
I20260807 18:16:01.513320 7434 raft_consensus.cc:504] T 00000000000000000000000000000000 P d800f8a66628417d808b2e4b0961b702 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d800f8a66628417d808b2e4b0961b702" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 39535 } }
I20260807 18:16:01.513972 7434 leader_election.cc:302] T 00000000000000000000000000000000 P d800f8a66628417d808b2e4b0961b702 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d800f8a66628417d808b2e4b0961b702; no voters:
I20260807 18:16:01.514648 7434 leader_election.cc:288] T 00000000000000000000000000000000 P d800f8a66628417d808b2e4b0961b702 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 18:16:01.514794 7441 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d800f8a66628417d808b2e4b0961b702 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 18:16:01.517270 7434 sys_catalog.cc:564] T 00000000000000000000000000000000 P d800f8a66628417d808b2e4b0961b702 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:16:01.517823 7441 raft_consensus.cc:686] T 00000000000000000000000000000000 P d800f8a66628417d808b2e4b0961b702 [term 2 LEADER]: Becoming Leader. State: Replica: d800f8a66628417d808b2e4b0961b702, State: Running, Role: LEADER
I20260807 18:16:01.518507 7441 consensus_queue.cc:237] T 00000000000000000000000000000000 P d800f8a66628417d808b2e4b0961b702 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d800f8a66628417d808b2e4b0961b702" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 39535 } }
I20260807 18:16:01.521977 7443 sys_catalog.cc:455] T 00000000000000000000000000000000 P d800f8a66628417d808b2e4b0961b702 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d800f8a66628417d808b2e4b0961b702. Latest consensus state: current_term: 2 leader_uuid: "d800f8a66628417d808b2e4b0961b702" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d800f8a66628417d808b2e4b0961b702" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 39535 } } }
I20260807 18:16:01.522379 7443 sys_catalog.cc:458] T 00000000000000000000000000000000 P d800f8a66628417d808b2e4b0961b702 [sys.catalog]: This master's current role is: LEADER
I20260807 18:16:01.523206 7442 sys_catalog.cc:455] T 00000000000000000000000000000000 P d800f8a66628417d808b2e4b0961b702 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "d800f8a66628417d808b2e4b0961b702" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d800f8a66628417d808b2e4b0961b702" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 39535 } } }
I20260807 18:16:01.523506 7442 sys_catalog.cc:458] T 00000000000000000000000000000000 P d800f8a66628417d808b2e4b0961b702 [sys.catalog]: This master's current role is: LEADER
I20260807 18:16:01.534843 7450 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:16:01.543241 7450 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:16:01.544902 7450 catalog_manager.cc:1259] Loaded cluster ID: 67ef141743fc47daa7707f2f49ced5ca
I20260807 18:16:01.545118 7450 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:16:01.548799 7450 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260807 18:16:01.550187 7461 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260807 18:16:01.566694 7450 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260807 18:16:01.577127 7450 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:16:01.579007 7450 catalog_manager.cc:5879] T 00000000000000000000000000000000 P d800f8a66628417d808b2e4b0961b702: Loaded TSK: 0
I20260807 18:16:01.580596 7450 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 18:16:01.845256 7438 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:16:01.845651 7438 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:16:01.845841 7438 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:16:01.853454 7438 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:16:01.853677 7438 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:16:01.853857 7438 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.154.1
I20260807 18:16:01.864111 7438 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.154.20:44041
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.1.keytab
--principal=kudu/127.18.154.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.154.1:44921
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.154.1
--webserver_port=36253
--tserver_master_addrs=127.18.154.62:39535
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.154.1
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:16:01.865509 7438 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:16:01.866995 7438 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:16:01.882912 7470 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:16:01.882933 7469 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:16:01.886334 7474 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:16:01.887106 7438 server_base.cc:1034] running on GCE node
I20260807 18:16:01.887802 7438 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:16:01.888928 7438 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:16:01.890192 7438 hybrid_clock.cc:648] HybridClock initialized: now 1786126561890120 us; error 51 us; skew 500 ppm
I20260807 18:16:01.890599 7438 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:16:01 dist-test-slave-gkw7 krb5kdc[6552](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126561, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:16:01.894225 7438 init.cc:377] Logged in from keytab as kudu/127.18.154.1@KRBTEST.COM (short username kudu)
I20260807 18:16:01.896301 7438 webserver.cc:460] Webserver started at http://127.18.154.1:36253/ using document root <none> and password file <none>
I20260807 18:16:01.897334 7438 fs_manager.cc:359] Metadata directory not provided
I20260807 18:16:01.897502 7438 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:16:01.905150 7438 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 18:16:01.908670 7486 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:16:01.910421 7438 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260807 18:16:01.910624 7438 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/ts-0/wal
uuid: "70281fa69b7949aead7a99ec552fa256"
format_stamp: "Formatted at 2026-08-07 18:15:58 on dist-test-slave-gkw7"
server_key: "e9b0db28c63883e21bb2301158686274"
server_key_iv: "c890fb8f75d03d6d75dcf07022b40aae"
server_key_version: "encryptionkey@0"
I20260807 18:16:01.911994 7438 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:16:01.927652 7438 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:16:01.932016 7438 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:16:01.932263 7438 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:16:01.933341 7438 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:16:01.934973 7438 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:16:01.935111 7438 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:16:01.935204 7438 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:16:01.935288 7438 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:16:01.972985 7438 rpc_server.cc:307] RPC server started. Bound to: 127.18.154.1:44921
I20260807 18:16:01.973152 7637 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.154.1:44921 every 8 connection(s)
I20260807 18:16:01.974762 7438 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/ts-0/data/info.pb
I20260807 18:16:01.981727 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 7438
I20260807 18:16:01.982452 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.154.2:45587
--local_ip_for_outbound_sockets=127.18.154.2
--tserver_master_addrs=127.18.154.62:39535
--webserver_port=35589
--webserver_interface=127.18.154.2
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.2.keytab
--principal=kudu/127.18.154.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.154.20:44041
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 18:16:01 dist-test-slave-gkw7 krb5kdc[6552](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126561, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.1@KRBTEST.COM for kudu/127.18.154.62@KRBTEST.COM
I20260807 18:16:02.006826 7643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:01.978498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:39535 (local address 127.18.154.1:50295)
0807 18:16:01.979313 (+ 815us) negotiation.cc:107] Waiting for socket to connect
0807 18:16:01.979366 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0807 18:16:01.981307 (+ 1941us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:16:01.983179 (+ 1872us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:16:01.983193 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:16:01.984061 (+ 868us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:16:01.985014 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:16:01.985045 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:16:01.987389 (+ 2344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:16:01.987397 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:16:01.989831 (+ 2434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:16:01.989852 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:16:01.991080 (+ 1228us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:16:01.991090 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:16:01.991417 (+ 327us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:16:01.992320 (+ 903us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:16:01.992362 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:16:01.995685 (+ 3323us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:16:02.001286 (+ 5601us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:16:02.001297 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:16:02.001312 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:16:02.002049 (+ 737us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:16:02.002753 (+ 704us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:16:02.002757 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:16:02.002760 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:16:02.002984 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:16:02.004360 (+ 1376us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:16:02.004367 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:16:02.004713 (+ 346us) client_negotiation.cc:769] Sending connection context
0807 18:16:02.004980 (+ 267us) client_negotiation.cc:240] Negotiation successful
0807 18:16:02.005610 (+ 630us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":567,"thread_start_us":325,"threads_started":1}
I20260807 18:16:02.007509 7642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:01.978209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.1:50295 (local address 127.18.154.62:39535)
0807 18:16:01.978936 (+ 727us) server_negotiation.cc:206] Beginning negotiation
0807 18:16:01.978956 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0807 18:16:01.979512 (+ 556us) server_negotiation.cc:407] Connection header received
0807 18:16:01.982088 (+ 2576us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:16:01.982106 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:16:01.982512 (+ 406us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:16:01.982943 (+ 431us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:16:01.985438 (+ 2495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:16:01.987186 (+ 1748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:16:01.990009 (+ 2823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:16:01.990807 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:16:01.990972 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:16:01.996070 (+ 5098us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:16:01.996087 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:16:01.996115 (+ 28us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:16:01.996201 (+ 86us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:16:02.001042 (+ 4841us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:16:02.002280 (+ 1238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:16:02.002294 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:16:02.002307 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:16:02.002588 (+ 281us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:16:02.003142 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:16:02.003160 (+ 18us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:16:02.003167 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:16:02.004024 (+ 857us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:16:02.004280 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0807 18:16:02.005362 (+ 1082us) server_negotiation.cc:299] Negotiation successful
0807 18:16:02.005965 (+ 603us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":468,"thread_start_us":329,"threads_started":1}
I20260807 18:16:02.010910 7640 heartbeater.cc:347] Connected to a master server at 127.18.154.62:39535
I20260807 18:16:02.011281 7640 heartbeater.cc:464] Registering TS with master...
I20260807 18:16:02.012148 7640 heartbeater.cc:511] Master 127.18.154.62:39535 requested a full tablet report, sending...
I20260807 18:16:02.014688 7386 ts_manager.cc:194] Registered new tserver with Master: 70281fa69b7949aead7a99ec552fa256 (127.18.154.1:44921)
I20260807 18:16:02.016346 7386 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.154.1@KRBTEST.COM'} at 127.18.154.1:50295
W20260807 18:16:02.352855 7644 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:16:02.353170 7644 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:16:02.353322 7644 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:16:02.360857 7644 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:16:02.360978 7644 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:16:02.361164 7644 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.154.2
I20260807 18:16:02.371613 7644 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.154.20:44041
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.2.keytab
--principal=kudu/127.18.154.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.154.2:45587
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.154.2
--webserver_port=35589
--tserver_master_addrs=127.18.154.62:39535
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.154.2
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:16:02.373050 7644 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:16:02.374513 7644 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:16:02.388762 7652 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:16:02.388870 7657 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:16:02.388835 7649 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:16:02.389881 7644 server_base.cc:1034] running on GCE node
I20260807 18:16:02.390604 7644 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:16:02.391620 7644 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:16:02.392887 7644 hybrid_clock.cc:648] HybridClock initialized: now 1786126562392814 us; error 70 us; skew 500 ppm
I20260807 18:16:02.393307 7644 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:16:02 dist-test-slave-gkw7 krb5kdc[6552](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126562, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:16:02.397099 7644 init.cc:377] Logged in from keytab as kudu/127.18.154.2@KRBTEST.COM (short username kudu)
I20260807 18:16:02.398743 7644 webserver.cc:460] Webserver started at http://127.18.154.2:35589/ using document root <none> and password file <none>
I20260807 18:16:02.399513 7644 fs_manager.cc:359] Metadata directory not provided
I20260807 18:16:02.399683 7644 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:16:02.405516 7644 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260807 18:16:02.408694 7669 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:16:02.410537 7644 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:16:02.410742 7644 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/ts-1/wal
uuid: "20725a0434e54ab9b7d73c8733499c30"
format_stamp: "Formatted at 2026-08-07 18:15:59 on dist-test-slave-gkw7"
server_key: "7b63ade0bc23bcb7a97748847d6b3f5a"
server_key_iv: "fec9806a8bfee1cbc4c4db5873e15e93"
server_key_version: "encryptionkey@0"
I20260807 18:16:02.411967 7644 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:16:02.438017 7644 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:16:02.442951 7644 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:16:02.443295 7644 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:16:02.444435 7644 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:16:02.446028 7644 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:16:02.446187 7644 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:16:02.446331 7644 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:16:02.446430 7644 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:16:02.482650 7644 rpc_server.cc:307] RPC server started. Bound to: 127.18.154.2:45587
I20260807 18:16:02.482686 7832 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.154.2:45587 every 8 connection(s)
I20260807 18:16:02.484331 7644 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/ts-1/data/info.pb
I20260807 18:16:02.490353 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 7644
I20260807 18:16:02.491006 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.154.3:44287
--local_ip_for_outbound_sockets=127.18.154.3
--tserver_master_addrs=127.18.154.62:39535
--webserver_port=38245
--webserver_interface=127.18.154.3
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.3.keytab
--principal=kudu/127.18.154.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.154.20:44041
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 18:16:02 dist-test-slave-gkw7 krb5kdc[6552](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126562, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.2@KRBTEST.COM for kudu/127.18.154.62@KRBTEST.COM
I20260807 18:16:02.506764 7642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:02.487574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.2:57687 (local address 127.18.154.62:39535)
0807 18:16:02.487785 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0807 18:16:02.487790 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:16:02.488985 (+ 1195us) server_negotiation.cc:407] Connection header received
0807 18:16:02.491075 (+ 2090us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:16:02.491083 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:16:02.491189 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:16:02.491301 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:16:02.492933 (+ 1632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:16:02.493702 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:16:02.495313 (+ 1611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:16:02.495741 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:16:02.495880 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:16:02.500125 (+ 4245us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:16:02.500134 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:16:02.500142 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:16:02.500193 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:16:02.503451 (+ 3258us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:16:02.504220 (+ 769us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:16:02.504224 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:16:02.504226 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:16:02.504374 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:16:02.504861 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:16:02.504868 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:16:02.504875 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:16:02.505159 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:16:02.505330 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0807 18:16:02.506134 (+ 804us) server_negotiation.cc:299] Negotiation successful
0807 18:16:02.506378 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260807 18:16:02.507997 7837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:02.488030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:39535 (local address 127.18.154.2:57687)
0807 18:16:02.488797 (+ 767us) negotiation.cc:107] Waiting for socket to connect
0807 18:16:02.488836 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 18:16:02.490771 (+ 1935us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:16:02.491581 (+ 810us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:16:02.491590 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:16:02.492135 (+ 545us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:16:02.492754 (+ 619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:16:02.492771 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:16:02.493861 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:16:02.493865 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:16:02.495159 (+ 1294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:16:02.495171 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:16:02.496005 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:16:02.496012 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:16:02.496255 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:16:02.497103 (+ 848us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:16:02.497138 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:16:02.499844 (+ 2706us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:16:02.503603 (+ 3759us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:16:02.503611 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:16:02.503627 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:16:02.504101 (+ 474us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:16:02.504515 (+ 414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:16:02.504520 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:16:02.504524 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:16:02.504745 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:16:02.505297 (+ 552us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:16:02.505306 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:16:02.505740 (+ 434us) client_negotiation.cc:769] Sending connection context
0807 18:16:02.506039 (+ 299us) client_negotiation.cc:240] Negotiation successful
0807 18:16:02.506668 (+ 629us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":517,"thread_start_us":292,"threads_started":1}
I20260807 18:16:02.509618 7834 heartbeater.cc:347] Connected to a master server at 127.18.154.62:39535
I20260807 18:16:02.510017 7834 heartbeater.cc:464] Registering TS with master...
I20260807 18:16:02.511142 7834 heartbeater.cc:511] Master 127.18.154.62:39535 requested a full tablet report, sending...
I20260807 18:16:02.512916 7386 ts_manager.cc:194] Registered new tserver with Master: 20725a0434e54ab9b7d73c8733499c30 (127.18.154.2:45587)
I20260807 18:16:02.513845 7386 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.154.2@KRBTEST.COM'} at 127.18.154.2:57687
W20260807 18:16:02.844806 7838 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:16:02.845094 7838 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:16:02.845285 7838 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:16:02.852573 7838 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:16:02.852715 7838 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:16:02.852908 7838 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.154.3
I20260807 18:16:02.862950 7838 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.154.20:44041
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.3.keytab
--principal=kudu/127.18.154.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.154.3:44287
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.154.3
--webserver_port=38245
--tserver_master_addrs=127.18.154.62:39535
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.154.3
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:16:02.864437 7838 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:16:02.865944 7838 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:16:02.880028 7846 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:16:02.879977 7847 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:16:02.880054 7851 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:16:02.880889 7838 server_base.cc:1034] running on GCE node
I20260807 18:16:02.881613 7838 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:16:02.882658 7838 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:16:02.883895 7838 hybrid_clock.cc:648] HybridClock initialized: now 1786126562883841 us; error 52 us; skew 500 ppm
I20260807 18:16:02.884236 7838 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:16:02 dist-test-slave-gkw7 krb5kdc[6552](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126562, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:16:02.887488 7838 init.cc:377] Logged in from keytab as kudu/127.18.154.3@KRBTEST.COM (short username kudu)
I20260807 18:16:02.889045 7838 webserver.cc:460] Webserver started at http://127.18.154.3:38245/ using document root <none> and password file <none>
I20260807 18:16:02.889889 7838 fs_manager.cc:359] Metadata directory not provided
I20260807 18:16:02.890055 7838 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:16:02.895709 7838 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 18:16:02.898737 7862 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:16:02.900252 7838 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:16:02.900457 7838 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/ts-2/wal
uuid: "9fc0d7d72d654b3fbed6817c5715dc59"
format_stamp: "Formatted at 2026-08-07 18:15:59 on dist-test-slave-gkw7"
server_key: "144ea1a0532ead184ac0c8c2989ee2a9"
server_key_iv: "ed37e496730df29551cea2c777ddc950"
server_key_version: "encryptionkey@0"
I20260807 18:16:02.901503 7838 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:16:02.917176 7838 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:16:02.921974 7838 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:16:02.922271 7838 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:16:02.923302 7838 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:16:02.924811 7838 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:16:02.924930 7838 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:16:02.925058 7838 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:16:02.925143 7838 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:16:02.956740 7838 rpc_server.cc:307] RPC server started. Bound to: 127.18.154.3:44287
I20260807 18:16:02.956804 8017 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.154.3:44287 every 8 connection(s)
I20260807 18:16:02.958352 7838 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/ts-2/data/info.pb
I20260807 18:16:02.965734 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 7838
Aug 07 18:16:02 dist-test-slave-gkw7 krb5kdc[6552](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126562, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.3@KRBTEST.COM for kudu/127.18.154.62@KRBTEST.COM
I20260807 18:16:02.982947 7642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:02.961108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.3:42207 (local address 127.18.154.62:39535)
0807 18:16:02.961305 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0807 18:16:02.961310 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:16:02.962678 (+ 1368us) server_negotiation.cc:407] Connection header received
0807 18:16:02.964527 (+ 1849us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:16:02.964533 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:16:02.964624 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:16:02.964735 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:16:02.966398 (+ 1663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:16:02.967428 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:16:02.969629 (+ 2201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:16:02.970097 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:16:02.970283 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:16:02.974865 (+ 4582us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:16:02.974874 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:16:02.974880 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:16:02.974922 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:16:02.979268 (+ 4346us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:16:02.980338 (+ 1070us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:16:02.980343 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:16:02.980345 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:16:02.980476 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:16:02.981011 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:16:02.981015 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:16:02.981017 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:16:02.981320 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:16:02.981497 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0807 18:16:02.982318 (+ 821us) server_negotiation.cc:299] Negotiation successful
0807 18:16:02.982561 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260807 18:16:02.983371 8021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:02.966697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50042 (local address 127.18.154.62:39535)
0807 18:16:02.967329 (+ 632us) server_negotiation.cc:206] Beginning negotiation
0807 18:16:02.967338 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:16:02.967859 (+ 521us) server_negotiation.cc:407] Connection header received
0807 18:16:02.968049 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:16:02.968058 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:16:02.968116 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:16:02.968224 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:16:02.970055 (+ 1831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:16:02.970863 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:16:02.972482 (+ 1619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:16:02.973150 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:16:02.973304 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:16:02.976855 (+ 3551us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:16:02.976860 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:16:02.976864 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:16:02.976912 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:16:02.980471 (+ 3559us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:16:02.981174 (+ 703us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:16:02.981182 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:16:02.981186 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:16:02.981309 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:16:02.981791 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:16:02.981798 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:16:02.981802 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:16:02.982077 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:16:02.982319 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0807 18:16:02.982666 (+ 347us) server_negotiation.cc:299] Negotiation successful
0807 18:16:02.982897 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":456,"thread_start_us":284,"threads_started":1}
I20260807 18:16:02.984419 8020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:02.961676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:39535 (local address 127.18.154.3:42207)
0807 18:16:02.962411 (+ 735us) negotiation.cc:107] Waiting for socket to connect
0807 18:16:02.962450 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 18:16:02.964244 (+ 1794us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:16:02.964980 (+ 736us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:16:02.964988 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:16:02.965522 (+ 534us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:16:02.966214 (+ 692us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:16:02.966231 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:16:02.967599 (+ 1368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:16:02.967605 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:16:02.969401 (+ 1796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:16:02.969415 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:16:02.970384 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:16:02.970396 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:16:02.970691 (+ 295us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:16:02.971478 (+ 787us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:16:02.971503 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:16:02.974584 (+ 3081us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:16:02.979457 (+ 4873us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:16:02.979468 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:16:02.979487 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:16:02.980184 (+ 697us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:16:02.980610 (+ 426us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:16:02.980616 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:16:02.980621 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:16:02.980861 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:16:02.981478 (+ 617us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:16:02.981487 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:16:02.981915 (+ 428us) client_negotiation.cc:769] Sending connection context
0807 18:16:02.982204 (+ 289us) client_negotiation.cc:240] Negotiation successful
0807 18:16:02.982820 (+ 616us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":490,"thread_start_us":279,"threads_started":1}
I20260807 18:16:02.986156 8018 heartbeater.cc:347] Connected to a master server at 127.18.154.62:39535
I20260807 18:16:02.986510 8018 heartbeater.cc:464] Registering TS with master...
I20260807 18:16:02.987367 8018 heartbeater.cc:511] Master 127.18.154.62:39535 requested a full tablet report, sending...
I20260807 18:16:02.988766 7386 ts_manager.cc:194] Registered new tserver with Master: 9fc0d7d72d654b3fbed6817c5715dc59 (127.18.154.3:44287)
I20260807 18:16:02.989719 7386 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.154.3@KRBTEST.COM'} at 127.18.154.3:42207
I20260807 18:16:02.999579 19048 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 18:16:03.007397 7642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:03.000688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50056 (local address 127.18.154.62:39535)
0807 18:16:03.000915 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0807 18:16:03.000923 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:16:03.001291 (+ 368us) server_negotiation.cc:407] Connection header received
0807 18:16:03.001578 (+ 287us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:16:03.001586 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:16:03.001739 (+ 153us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:16:03.001908 (+ 169us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:16:03.002596 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:16:03.003383 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:16:03.005004 (+ 1621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:16:03.005439 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:16:03.005665 (+ 226us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:16:03.006200 (+ 535us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:16:03.006817 (+ 617us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:16:03.006974 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0807 18:16:03.007036 (+ 62us) server_negotiation.cc:299] Negotiation successful
0807 18:16:03.007127 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 18:16:03.008404 7386 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:50056:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 18:16:03.016155 7386 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:16:03.020962 7640 heartbeater.cc:503] Master 127.18.154.62:39535 was elected leader, sending a full tablet report...
I20260807 18:16:03.051733 8036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:03.029928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54520 (local address 127.18.154.3:44287)
0807 18:16:03.040777 (+ 10849us) server_negotiation.cc:206] Beginning negotiation
0807 18:16:03.040790 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 18:16:03.041918 (+ 1128us) server_negotiation.cc:407] Connection header received
0807 18:16:03.042056 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:16:03.042065 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:16:03.042360 (+ 295us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:16:03.043743 (+ 1383us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:16:03.044104 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:16:03.046381 (+ 2277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:16:03.048987 (+ 2606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:16:03.050580 (+ 1593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:16:03.050747 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:16:03.050979 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0807 18:16:03.051216 (+ 237us) server_negotiation.cc:299] Negotiation successful
0807 18:16:03.051379 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10630,"thread_start_us":10431,"threads_started":1}
I20260807 18:16:03.051740 8034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:03.029624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.3:44287 (local address 127.0.0.1:54520)
0807 18:16:03.041283 (+ 11659us) negotiation.cc:107] Waiting for socket to connect
0807 18:16:03.041345 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0807 18:16:03.041596 (+ 251us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:16:03.042680 (+ 1084us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:16:03.042692 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:16:03.042742 (+ 50us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:16:03.043441 (+ 699us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:16:03.043460 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:16:03.046568 (+ 3108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:16:03.046574 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:16:03.048741 (+ 2167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:16:03.048756 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:16:03.050817 (+ 2061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:16:03.050824 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:16:03.051006 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:16:03.051043 (+ 37us) client_negotiation.cc:769] Sending connection context
0807 18:16:03.051153 (+ 110us) client_negotiation.cc:240] Negotiation successful
0807 18:16:03.051291 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11391,"thread_start_us":11217,"threads_started":1}
I20260807 18:16:03.051810 8037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:03.030093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.2:45587 (local address 127.0.0.1:60906)
0807 18:16:03.043391 (+ 13298us) negotiation.cc:107] Waiting for socket to connect
0807 18:16:03.043409 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 18:16:03.043574 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:16:03.044133 (+ 559us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:16:03.044138 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:16:03.044164 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:16:03.044503 (+ 339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:16:03.044512 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:16:03.045889 (+ 1377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:16:03.045894 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:16:03.048196 (+ 2302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:16:03.048216 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:16:03.050302 (+ 2086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:16:03.050313 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:16:03.050604 (+ 291us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:16:03.050683 (+ 79us) client_negotiation.cc:769] Sending connection context
0807 18:16:03.050872 (+ 189us) client_negotiation.cc:240] Negotiation successful
0807 18:16:03.051163 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13151,"mutex_wait_us":4,"thread_start_us":12987,"threads_started":1}
I20260807 18:16:03.053251 8038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:03.030326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60906 (local address 127.18.154.2:45587)
0807 18:16:03.042579 (+ 12253us) server_negotiation.cc:206] Beginning negotiation
0807 18:16:03.042593 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 18:16:03.043529 (+ 936us) server_negotiation.cc:407] Connection header received
0807 18:16:03.043755 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:16:03.043767 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:16:03.043950 (+ 183us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:16:03.044155 (+ 205us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:16:03.044664 (+ 509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:16:03.045756 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:16:03.048404 (+ 2648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:16:03.050042 (+ 1638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:16:03.050185 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:16:03.050401 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0807 18:16:03.052488 (+ 2087us) server_negotiation.cc:299] Negotiation successful
0807 18:16:03.052747 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12028,"thread_start_us":12259,"threads_started":1}
I20260807 18:16:03.057437 7740 tablet_service.cc:1467] Processing CreateTablet for tablet b4b6cd2b154d4e3aa93d7e5eebf0f761 (DEFAULT_TABLE table=test-table [id=adb21dd62ee640179b6e165dce456cc7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:16:03.057700 7924 tablet_service.cc:1467] Processing CreateTablet for tablet b4b6cd2b154d4e3aa93d7e5eebf0f761 (DEFAULT_TABLE table=test-table [id=adb21dd62ee640179b6e165dce456cc7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:16:03.059092 7740 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b4b6cd2b154d4e3aa93d7e5eebf0f761. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:16:03.059090 7924 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b4b6cd2b154d4e3aa93d7e5eebf0f761. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:16:03.060385 8039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:03.042257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58334 (local address 127.18.154.1:44921)
0807 18:16:03.050213 (+ 7956us) server_negotiation.cc:206] Beginning negotiation
0807 18:16:03.050222 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:16:03.052297 (+ 2075us) server_negotiation.cc:407] Connection header received
0807 18:16:03.052419 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:16:03.052429 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:16:03.052713 (+ 284us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:16:03.052933 (+ 220us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:16:03.053310 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:16:03.055833 (+ 2523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:16:03.057342 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:16:03.058808 (+ 1466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:16:03.058968 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:16:03.059210 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0807 18:16:03.060013 (+ 803us) server_negotiation.cc:299] Negotiation successful
0807 18:16:03.060148 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7770,"thread_start_us":7611,"threads_started":1}
I20260807 18:16:03.059834 8034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:03.046791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.1:44921 (local address 127.0.0.1:58334)
0807 18:16:03.052039 (+ 5248us) negotiation.cc:107] Waiting for socket to connect
0807 18:16:03.052059 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 18:16:03.052168 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:16:03.052907 (+ 739us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:16:03.052912 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:16:03.052940 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:16:03.053145 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:16:03.053160 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:16:03.055979 (+ 2819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:16:03.055986 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:16:03.057161 (+ 1175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:16:03.057173 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:16:03.058978 (+ 1805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:16:03.058984 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:16:03.059236 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:16:03.059278 (+ 42us) client_negotiation.cc:769] Sending connection context
0807 18:16:03.059377 (+ 99us) client_negotiation.cc:240] Negotiation successful
0807 18:16:03.059521 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5102,"thread_start_us":6873,"threads_started":1}
I20260807 18:16:03.064631 7552 tablet_service.cc:1467] Processing CreateTablet for tablet b4b6cd2b154d4e3aa93d7e5eebf0f761 (DEFAULT_TABLE table=test-table [id=adb21dd62ee640179b6e165dce456cc7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:16:03.066000 7552 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b4b6cd2b154d4e3aa93d7e5eebf0f761. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:16:03.081167 8043 tablet_bootstrap.cc:492] T b4b6cd2b154d4e3aa93d7e5eebf0f761 P 20725a0434e54ab9b7d73c8733499c30: Bootstrap starting.
I20260807 18:16:03.081499 8042 tablet_bootstrap.cc:492] T b4b6cd2b154d4e3aa93d7e5eebf0f761 P 9fc0d7d72d654b3fbed6817c5715dc59: Bootstrap starting.
I20260807 18:16:03.081842 8045 tablet_bootstrap.cc:492] T b4b6cd2b154d4e3aa93d7e5eebf0f761 P 70281fa69b7949aead7a99ec552fa256: Bootstrap starting.
I20260807 18:16:03.085176 8042 tablet_bootstrap.cc:654] T b4b6cd2b154d4e3aa93d7e5eebf0f761 P 9fc0d7d72d654b3fbed6817c5715dc59: Neither blocks nor log segments found. Creating new log.
I20260807 18:16:03.085593 8045 tablet_bootstrap.cc:654] T b4b6cd2b154d4e3aa93d7e5eebf0f761 P 70281fa69b7949aead7a99ec552fa256: Neither blocks nor log segments found. Creating new log.
I20260807 18:16:03.085614 8043 tablet_bootstrap.cc:654] T b4b6cd2b154d4e3aa93d7e5eebf0f761 P 20725a0434e54ab9b7d73c8733499c30: Neither blocks nor log segments found. Creating new log.
I20260807 18:16:03.086884 8045 log.cc:824] T b4b6cd2b154d4e3aa93d7e5eebf0f761 P 70281fa69b7949aead7a99ec552fa256: Log is configured to *not* fsync() on all Append() calls
I20260807 18:16:03.086884 8043 log.cc:824] T b4b6cd2b154d4e3aa93d7e5eebf0f761 P 20725a0434e54ab9b7d73c8733499c30: Log is configured to *not* fsync() on all Append() calls
I20260807 18:16:03.087218 8042 log.cc:824] T b4b6cd2b154d4e3aa93d7e5eebf0f761 P 9fc0d7d72d654b3fbed6817c5715dc59: Log is configured to *not* fsync() on all Append() calls
I20260807 18:16:03.090698 8045 tablet_bootstrap.cc:492] T b4b6cd2b154d4e3aa93d7e5eebf0f761 P 70281fa69b7949aead7a99ec552fa256: No bootstrap required, opened a new log
I20260807 18:16:03.090821 8042 tablet_bootstrap.cc:492] T b4b6cd2b154d4e3aa93d7e5eebf0f761 P 9fc0d7d72d654b3fbed6817c5715dc59: No bootstrap required, opened a new log
I20260807 18:16:03.091037 8045 ts_tablet_manager.cc:1397] T b4b6cd2b154d4e3aa93d7e5eebf0f761 P 70281fa69b7949aead7a99ec552fa256: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260807 18:16:03.091101 8042 ts_tablet_manager.cc:1397] T b4b6cd2b154d4e3aa93d7e5eebf0f761 P 9fc0d7d72d654b3fbed6817c5715dc59: Time spent bootstrapping tablet: real 0.013s user 0.005s sys 0.002s
I20260807 18:16:03.091537 8043 tablet_bootstrap.cc:492] T b4b6cd2b154d4e3aa93d7e5eebf0f761 P 20725a0434e54ab9b7d73c8733499c30: No bootstrap required, opened a new log
I20260807 18:16:03.091809 8043 ts_tablet_manager.cc:1397] T b4b6cd2b154d4e3aa93d7e5eebf0f761 P 20725a0434e54ab9b7d73c8733499c30: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.000s
I20260807 18:16:03.098963 8045 raft_consensus.cc:348] T b4b6cd2b154d4e3aa93d7e5eebf0f761 P 70281fa69b7949aead7a99ec552fa256 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fc0d7d72d654b3fbed6817c5715dc59" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 44287 } } peers { permanent_uuid: "70281fa69b7949aead7a99ec552fa256" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 44921 } } peers { permanent_uuid: "20725a0434e54ab9b7d73c8733499c30" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 45587 } }
I20260807 18:16:03.099373 8045 raft_consensus.cc:374] T b4b6cd2b154d4e3aa93d7e5eebf0f761 P 70281fa69b7949aead7a99ec552fa256 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:16:03.099488 8045 raft_consensus.cc:729] T b4b6cd2b154d4e3aa93d7e5eebf0f761 P 70281fa69b7949aead7a99ec552fa256 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 70281fa69b7949aead7a99ec552fa256, State: Initialized, Role: FOLLOWER
I20260807 18:16:03.100116 8045 consensus_queue.cc:260] T b4b6cd2b154d4e3aa93d7e5eebf0f761 P 70281fa69b7949aead7a99ec552fa256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fc0d7d72d654b3fbed6817c5715dc59" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 44287 } } peers { permanent_uuid: "70281fa69b7949aead7a99ec552fa256" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 44921 } } peers { permanent_uuid: "20725a0434e54ab9b7d73c8733499c30" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 45587 } }
I20260807 18:16:03.101998 8045 ts_tablet_manager.cc:1428] T b4b6cd2b154d4e3aa93d7e5eebf0f761 P 70281fa69b7949aead7a99ec552fa256: Time spent starting tablet: real 0.011s user 0.012s sys 0.000s
I20260807 18:16:03.101963 8042 raft_consensus.cc:348] T b4b6cd2b154d4e3aa93d7e5eebf0f761 P 9fc0d7d72d654b3fbed6817c5715dc59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fc0d7d72d654b3fbed6817c5715dc59" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 44287 } } peers { permanent_uuid: "70281fa69b7949aead7a99ec552fa256" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 44921 } } peers { permanent_uuid: "20725a0434e54ab9b7d73c8733499c30" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 45587 } }
I20260807 18:16:03.102415 8042 raft_consensus.cc:374] T b4b6cd2b154d4e3aa93d7e5eebf0f761 P 9fc0d7d72d654b3fbed6817c5715dc59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:16:03.102526 8042 raft_consensus.cc:729] T b4b6cd2b154d4e3aa93d7e5eebf0f761 P 9fc0d7d72d654b3fbed6817c5715dc59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9fc0d7d72d654b3fbed6817c5715dc59, State: Initialized, Role: FOLLOWER
I20260807 18:16:03.102415 8043 raft_consensus.cc:348] T b4b6cd2b154d4e3aa93d7e5eebf0f761 P 20725a0434e54ab9b7d73c8733499c30 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fc0d7d72d654b3fbed6817c5715dc59" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 44287 } } peers { permanent_uuid: "70281fa69b7949aead7a99ec552fa256" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 44921 } } peers { permanent_uuid: "20725a0434e54ab9b7d73c8733499c30" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 45587 } }
I20260807 18:16:03.102818 8043 raft_consensus.cc:374] T b4b6cd2b154d4e3aa93d7e5eebf0f761 P 20725a0434e54ab9b7d73c8733499c30 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:16:03.102972 8043 raft_consensus.cc:729] T b4b6cd2b154d4e3aa93d7e5eebf0f761 P 20725a0434e54ab9b7d73c8733499c30 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20725a0434e54ab9b7d73c8733499c30, State: Initialized, Role: FOLLOWER
I20260807 18:16:03.103228 8042 consensus_queue.cc:260] T b4b6cd2b154d4e3aa93d7e5eebf0f761 P 9fc0d7d72d654b3fbed6817c5715dc59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fc0d7d72d654b3fbed6817c5715dc59" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 44287 } } peers { permanent_uuid: "70281fa69b7949aead7a99ec552fa256" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 44921 } } peers { permanent_uuid: "20725a0434e54ab9b7d73c8733499c30" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 45587 } }
I20260807 18:16:03.103543 8043 consensus_queue.cc:260] T b4b6cd2b154d4e3aa93d7e5eebf0f761 P 20725a0434e54ab9b7d73c8733499c30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fc0d7d72d654b3fbed6817c5715dc59" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 44287 } } peers { permanent_uuid: "70281fa69b7949aead7a99ec552fa256" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 44921 } } peers { permanent_uuid: "20725a0434e54ab9b7d73c8733499c30" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 45587 } }
I20260807 18:16:03.105571 7834 heartbeater.cc:503] Master 127.18.154.62:39535 was elected leader, sending a full tablet report...
I20260807 18:16:03.105837 8043 ts_tablet_manager.cc:1428] T b4b6cd2b154d4e3aa93d7e5eebf0f761 P 20725a0434e54ab9b7d73c8733499c30: Time spent starting tablet: real 0.014s user 0.014s sys 0.001s
I20260807 18:16:03.109930 8018 heartbeater.cc:503] Master 127.18.154.62:39535 was elected leader, sending a full tablet report...
I20260807 18:16:03.109930 8042 ts_tablet_manager.cc:1428] T b4b6cd2b154d4e3aa93d7e5eebf0f761 P 9fc0d7d72d654b3fbed6817c5715dc59: Time spent starting tablet: real 0.019s user 0.011s sys 0.008s
W20260807 18:16:03.210140 8019 tablet.cc:2369] T b4b6cd2b154d4e3aa93d7e5eebf0f761 P 9fc0d7d72d654b3fbed6817c5715dc59: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 18:16:03.227579 7641 tablet.cc:2369] T b4b6cd2b154d4e3aa93d7e5eebf0f761 P 70281fa69b7949aead7a99ec552fa256: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 18:16:03.236804 7835 tablet.cc:2369] T b4b6cd2b154d4e3aa93d7e5eebf0f761 P 20725a0434e54ab9b7d73c8733499c30: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:16:03.237700 8056 raft_consensus.cc:482] T b4b6cd2b154d4e3aa93d7e5eebf0f761 P 20725a0434e54ab9b7d73c8733499c30 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:16:03.238013 8056 raft_consensus.cc:504] T b4b6cd2b154d4e3aa93d7e5eebf0f761 P 20725a0434e54ab9b7d73c8733499c30 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fc0d7d72d654b3fbed6817c5715dc59" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 44287 } } peers { permanent_uuid: "70281fa69b7949aead7a99ec552fa256" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 44921 } } peers { permanent_uuid: "20725a0434e54ab9b7d73c8733499c30" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 45587 } }
I20260807 18:16:03.239846 8056 leader_election.cc:288] T b4b6cd2b154d4e3aa93d7e5eebf0f761 P 20725a0434e54ab9b7d73c8733499c30 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9fc0d7d72d654b3fbed6817c5715dc59 (127.18.154.3:44287), 70281fa69b7949aead7a99ec552fa256 (127.18.154.1:44921)
I20260807 18:16:03.246464 8036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:03.240250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.2:46713 (local address 127.18.154.3:44287)
0807 18:16:03.240550 (+ 300us) server_negotiation.cc:206] Beginning negotiation
0807 18:16:03.240560 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:16:03.240948 (+ 388us) server_negotiation.cc:407] Connection header received
0807 18:16:03.241178 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:16:03.241183 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:16:03.241281 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:16:03.241390 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:16:03.242118 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:16:03.242858 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:16:03.244638 (+ 1780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:16:03.245450 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:16:03.245688 (+ 238us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:16:03.245768 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0807 18:16:03.246053 (+ 285us) server_negotiation.cc:299] Negotiation successful
0807 18:16:03.246196 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260807 18:16:03.246551 8057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:03.240081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.3:44287 (local address 127.18.154.2:46713)
0807 18:16:03.240817 (+ 736us) negotiation.cc:107] Waiting for socket to connect
0807 18:16:03.240835 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 18:16:03.241025 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:16:03.241522 (+ 497us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:16:03.241574 (+ 52us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:16:03.241606 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:16:03.241953 (+ 347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:16:03.241962 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:16:03.242986 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:16:03.242994 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:16:03.244483 (+ 1489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:16:03.244495 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:16:03.245710 (+ 1215us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:16:03.245718 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:16:03.245889 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:16:03.245914 (+ 25us) client_negotiation.cc:769] Sending connection context
0807 18:16:03.246022 (+ 108us) client_negotiation.cc:240] Negotiation successful
0807 18:16:03.246162 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":610,"thread_start_us":393,"threads_started":1}
I20260807 18:16:03.246990 8039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:03.241196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.2:54465 (local address 127.18.154.1:44921)
0807 18:16:03.241421 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0807 18:16:03.241427 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:16:03.241781 (+ 354us) server_negotiation.cc:407] Connection header received
0807 18:16:03.242001 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:16:03.242006 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:16:03.242068 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:16:03.242162 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:16:03.242761 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:16:03.243479 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:16:03.245200 (+ 1721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:16:03.246039 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:16:03.246217 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:16:03.246301 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0807 18:16:03.246604 (+ 303us) server_negotiation.cc:299] Negotiation successful
0807 18:16:03.246742 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260807 18:16:03.247028 8058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:03.240920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.1:44921 (local address 127.18.154.2:54465)
0807 18:16:03.241651 (+ 731us) negotiation.cc:107] Waiting for socket to connect
0807 18:16:03.241676 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0807 18:16:03.241843 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:16:03.242188 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:16:03.242193 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:16:03.242218 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:16:03.242577 (+ 359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:16:03.242586 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:16:03.243618 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:16:03.243624 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:16:03.245036 (+ 1412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:16:03.245048 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:16:03.246269 (+ 1221us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:16:03.246276 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:16:03.246443 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:16:03.246466 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 18:16:03.246577 (+ 111us) client_negotiation.cc:240] Negotiation successful
0807 18:16:03.246702 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":576,"thread_start_us":383,"threads_started":1}
I20260807 18:16:03.247936 7954 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4b6cd2b154d4e3aa93d7e5eebf0f761" candidate_uuid: "20725a0434e54ab9b7d73c8733499c30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9fc0d7d72d654b3fbed6817c5715dc59" is_pre_election: true
I20260807 18:16:03.248340 7583 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4b6cd2b154d4e3aa93d7e5eebf0f761" candidate_uuid: "20725a0434e54ab9b7d73c8733499c30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "70281fa69b7949aead7a99ec552fa256" is_pre_election: true
I20260807 18:16:03.248502 7954 raft_consensus.cc:2454] T b4b6cd2b154d4e3aa93d7e5eebf0f761 P 9fc0d7d72d654b3fbed6817c5715dc59 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20725a0434e54ab9b7d73c8733499c30 in term 0.
I20260807 18:16:03.248998 7583 raft_consensus.cc:2454] T b4b6cd2b154d4e3aa93d7e5eebf0f761 P 70281fa69b7949aead7a99ec552fa256 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20725a0434e54ab9b7d73c8733499c30 in term 0.
I20260807 18:16:03.249235 7673 leader_election.cc:302] T b4b6cd2b154d4e3aa93d7e5eebf0f761 P 20725a0434e54ab9b7d73c8733499c30 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20725a0434e54ab9b7d73c8733499c30, 9fc0d7d72d654b3fbed6817c5715dc59; no voters:
I20260807 18:16:03.249708 8056 raft_consensus.cc:2781] T b4b6cd2b154d4e3aa93d7e5eebf0f761 P 20725a0434e54ab9b7d73c8733499c30 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:16:03.249980 8056 raft_consensus.cc:482] T b4b6cd2b154d4e3aa93d7e5eebf0f761 P 20725a0434e54ab9b7d73c8733499c30 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:16:03.250126 8056 raft_consensus.cc:3037] T b4b6cd2b154d4e3aa93d7e5eebf0f761 P 20725a0434e54ab9b7d73c8733499c30 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:16:03.252555 8056 raft_consensus.cc:504] T b4b6cd2b154d4e3aa93d7e5eebf0f761 P 20725a0434e54ab9b7d73c8733499c30 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fc0d7d72d654b3fbed6817c5715dc59" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 44287 } } peers { permanent_uuid: "70281fa69b7949aead7a99ec552fa256" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 44921 } } peers { permanent_uuid: "20725a0434e54ab9b7d73c8733499c30" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 45587 } }
I20260807 18:16:03.253257 8056 leader_election.cc:288] T b4b6cd2b154d4e3aa93d7e5eebf0f761 P 20725a0434e54ab9b7d73c8733499c30 [CANDIDATE]: Term 1 election: Requested vote from peers 9fc0d7d72d654b3fbed6817c5715dc59 (127.18.154.3:44287), 70281fa69b7949aead7a99ec552fa256 (127.18.154.1:44921)
I20260807 18:16:03.253777 7954 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4b6cd2b154d4e3aa93d7e5eebf0f761" candidate_uuid: "20725a0434e54ab9b7d73c8733499c30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9fc0d7d72d654b3fbed6817c5715dc59"
I20260807 18:16:03.253901 7583 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4b6cd2b154d4e3aa93d7e5eebf0f761" candidate_uuid: "20725a0434e54ab9b7d73c8733499c30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "70281fa69b7949aead7a99ec552fa256"
I20260807 18:16:03.254012 7954 raft_consensus.cc:3037] T b4b6cd2b154d4e3aa93d7e5eebf0f761 P 9fc0d7d72d654b3fbed6817c5715dc59 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:16:03.254115 7583 raft_consensus.cc:3037] T b4b6cd2b154d4e3aa93d7e5eebf0f761 P 70281fa69b7949aead7a99ec552fa256 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:16:03.257354 7954 raft_consensus.cc:2454] T b4b6cd2b154d4e3aa93d7e5eebf0f761 P 9fc0d7d72d654b3fbed6817c5715dc59 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20725a0434e54ab9b7d73c8733499c30 in term 1.
I20260807 18:16:03.257354 7583 raft_consensus.cc:2454] T b4b6cd2b154d4e3aa93d7e5eebf0f761 P 70281fa69b7949aead7a99ec552fa256 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20725a0434e54ab9b7d73c8733499c30 in term 1.
I20260807 18:16:03.257769 7673 leader_election.cc:302] T b4b6cd2b154d4e3aa93d7e5eebf0f761 P 20725a0434e54ab9b7d73c8733499c30 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20725a0434e54ab9b7d73c8733499c30, 70281fa69b7949aead7a99ec552fa256; no voters:
I20260807 18:16:03.258064 8056 raft_consensus.cc:2781] T b4b6cd2b154d4e3aa93d7e5eebf0f761 P 20725a0434e54ab9b7d73c8733499c30 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:16:03.258538 8056 raft_consensus.cc:686] T b4b6cd2b154d4e3aa93d7e5eebf0f761 P 20725a0434e54ab9b7d73c8733499c30 [term 1 LEADER]: Becoming Leader. State: Replica: 20725a0434e54ab9b7d73c8733499c30, State: Running, Role: LEADER
I20260807 18:16:03.259021 8056 consensus_queue.cc:237] T b4b6cd2b154d4e3aa93d7e5eebf0f761 P 20725a0434e54ab9b7d73c8733499c30 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fc0d7d72d654b3fbed6817c5715dc59" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 44287 } } peers { permanent_uuid: "70281fa69b7949aead7a99ec552fa256" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 44921 } } peers { permanent_uuid: "20725a0434e54ab9b7d73c8733499c30" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 45587 } }
I20260807 18:16:03.263725 7383 catalog_manager.cc:5495] T b4b6cd2b154d4e3aa93d7e5eebf0f761 P 20725a0434e54ab9b7d73c8733499c30 reported cstate change: term changed from 0 to 1, leader changed from <none> to 20725a0434e54ab9b7d73c8733499c30 (127.18.154.2). New cstate: current_term: 1 leader_uuid: "20725a0434e54ab9b7d73c8733499c30" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fc0d7d72d654b3fbed6817c5715dc59" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 44287 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "70281fa69b7949aead7a99ec552fa256" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 44921 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20725a0434e54ab9b7d73c8733499c30" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 45587 } health_report { overall_health: HEALTHY } } }
I20260807 18:16:03.309044 8038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:03.303300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60922 (local address 127.18.154.2:45587)
0807 18:16:03.303533 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0807 18:16:03.303541 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:16:03.303601 (+ 60us) server_negotiation.cc:407] Connection header received
0807 18:16:03.303867 (+ 266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:16:03.303872 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:16:03.303971 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:16:03.304133 (+ 162us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:16:03.304548 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:16:03.305359 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:16:03.306843 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:16:03.307258 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:16:03.307393 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:16:03.307917 (+ 524us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:16:03.308446 (+ 529us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:16:03.308614 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0807 18:16:03.308689 (+ 75us) server_negotiation.cc:299] Negotiation successful
0807 18:16:03.308778 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260807 18:16:03.318315 7583 raft_consensus.cc:1261] T b4b6cd2b154d4e3aa93d7e5eebf0f761 P 70281fa69b7949aead7a99ec552fa256 [term 1 FOLLOWER]: Refusing update from remote peer 20725a0434e54ab9b7d73c8733499c30: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:16:03.318490 7954 raft_consensus.cc:1261] T b4b6cd2b154d4e3aa93d7e5eebf0f761 P 9fc0d7d72d654b3fbed6817c5715dc59 [term 1 FOLLOWER]: Refusing update from remote peer 20725a0434e54ab9b7d73c8733499c30: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:16:03.319406 8056 consensus_queue.cc:1035] T b4b6cd2b154d4e3aa93d7e5eebf0f761 P 20725a0434e54ab9b7d73c8733499c30 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70281fa69b7949aead7a99ec552fa256" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 44921 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:16:03.319717 8059 consensus_queue.cc:1035] T b4b6cd2b154d4e3aa93d7e5eebf0f761 P 20725a0434e54ab9b7d73c8733499c30 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9fc0d7d72d654b3fbed6817c5715dc59" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 44287 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:16:03.347359 7386 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:50056:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:16:03.347666 7386 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:50056:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:16:03.351390 7386 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d800f8a66628417d808b2e4b0961b702: Sending DeleteTablet for 3 replicas of tablet b4b6cd2b154d4e3aa93d7e5eebf0f761
I20260807 18:16:03.352782 7924 tablet_service.cc:1514] Processing DeleteTablet for tablet b4b6cd2b154d4e3aa93d7e5eebf0f761 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:16:03 UTC) from {username='kudu', principal='kudu/127.18.154.62@KRBTEST.COM'} at 127.0.0.1:54520
I20260807 18:16:03.352897 7740 tablet_service.cc:1514] Processing DeleteTablet for tablet b4b6cd2b154d4e3aa93d7e5eebf0f761 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:16:03 UTC) from {username='kudu', principal='kudu/127.18.154.62@KRBTEST.COM'} at 127.0.0.1:60906
I20260807 18:16:03.353302 7552 tablet_service.cc:1514] Processing DeleteTablet for tablet b4b6cd2b154d4e3aa93d7e5eebf0f761 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:16:03 UTC) from {username='kudu', principal='kudu/127.18.154.62@KRBTEST.COM'} at 127.0.0.1:58334
I20260807 18:16:03.353890 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 7438
I20260807 18:16:03.363922 8077 tablet_replica.cc:330] T b4b6cd2b154d4e3aa93d7e5eebf0f761 P 9fc0d7d72d654b3fbed6817c5715dc59: stopping tablet replica
I20260807 18:16:03.364442 8077 raft_consensus.cc:2229] T b4b6cd2b154d4e3aa93d7e5eebf0f761 P 9fc0d7d72d654b3fbed6817c5715dc59 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:16:03.365067 8077 raft_consensus.cc:2258] T b4b6cd2b154d4e3aa93d7e5eebf0f761 P 9fc0d7d72d654b3fbed6817c5715dc59 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:16:03.367568 8078 tablet_replica.cc:330] T b4b6cd2b154d4e3aa93d7e5eebf0f761 P 20725a0434e54ab9b7d73c8733499c30: stopping tablet replica
I20260807 18:16:03.368309 8078 raft_consensus.cc:2229] T b4b6cd2b154d4e3aa93d7e5eebf0f761 P 20725a0434e54ab9b7d73c8733499c30 [term 1 LEADER]: Raft consensus shutting down.
I20260807 18:16:03.369251 8078 raft_consensus.cc:2258] T b4b6cd2b154d4e3aa93d7e5eebf0f761 P 20725a0434e54ab9b7d73c8733499c30 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:16:03.371937 8078 ts_tablet_manager.cc:1905] T b4b6cd2b154d4e3aa93d7e5eebf0f761 P 20725a0434e54ab9b7d73c8733499c30: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:16:03.372222 8077 ts_tablet_manager.cc:1905] T b4b6cd2b154d4e3aa93d7e5eebf0f761 P 9fc0d7d72d654b3fbed6817c5715dc59: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:16:03.374115 8039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:03.354764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58340 (local address 127.18.154.1:44921)
0807 18:16:03.354945 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0807 18:16:03.354952 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:16:03.355158 (+ 206us) server_negotiation.cc:407] Connection header received
0807 18:16:03.355383 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:16:03.355388 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:16:03.355489 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:16:03.355599 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:16:03.357854 (+ 2255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:16:03.359079 (+ 1225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:16:03.361664 (+ 2585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:16:03.362403 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:16:03.362531 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:16:03.366629 (+ 4098us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:16:03.366648 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:16:03.366669 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:16:03.366711 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:16:03.370819 (+ 4108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:16:03.371736 (+ 917us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:16:03.371745 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:16:03.371749 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:16:03.371922 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:16:03.372333 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:16:03.372338 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:16:03.372342 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:16:03.372813 (+ 471us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:16:03.373132 (+ 319us) server_negotiation.cc:1035] Waiting for connection context
0807 18:16:03.373436 (+ 304us) server_negotiation.cc:299] Negotiation successful
0807 18:16:03.373709 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260807 18:16:03.375140 7633 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:16:03.381633 8077 ts_tablet_manager.cc:1918] T b4b6cd2b154d4e3aa93d7e5eebf0f761 P 9fc0d7d72d654b3fbed6817c5715dc59: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:16:03.381913 8077 log.cc:1196] T b4b6cd2b154d4e3aa93d7e5eebf0f761 P 9fc0d7d72d654b3fbed6817c5715dc59: Deleting WAL directory at /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/ts-2/wal/wals/b4b6cd2b154d4e3aa93d7e5eebf0f761
I20260807 18:16:03.382792 8077 ts_tablet_manager.cc:1939] T b4b6cd2b154d4e3aa93d7e5eebf0f761 P 9fc0d7d72d654b3fbed6817c5715dc59: Deleting consensus metadata
I20260807 18:16:03.385447 7366 catalog_manager.cc:4841] TS 9fc0d7d72d654b3fbed6817c5715dc59 (127.18.154.3:44287): tablet b4b6cd2b154d4e3aa93d7e5eebf0f761 (table test-table [id=adb21dd62ee640179b6e165dce456cc7]) successfully deleted
I20260807 18:16:03.391688 8078 ts_tablet_manager.cc:1918] T b4b6cd2b154d4e3aa93d7e5eebf0f761 P 20725a0434e54ab9b7d73c8733499c30: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:16:03.392091 8078 log.cc:1196] T b4b6cd2b154d4e3aa93d7e5eebf0f761 P 20725a0434e54ab9b7d73c8733499c30: Deleting WAL directory at /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/ts-1/wal/wals/b4b6cd2b154d4e3aa93d7e5eebf0f761
I20260807 18:16:03.392939 8078 ts_tablet_manager.cc:1939] T b4b6cd2b154d4e3aa93d7e5eebf0f761 P 20725a0434e54ab9b7d73c8733499c30: Deleting consensus metadata
I20260807 18:16:03.395022 7364 catalog_manager.cc:4841] TS 20725a0434e54ab9b7d73c8733499c30 (127.18.154.2:45587): tablet b4b6cd2b154d4e3aa93d7e5eebf0f761 (table test-table [id=adb21dd62ee640179b6e165dce456cc7]) successfully deleted
I20260807 18:16:03.480036 8079 tablet_replica.cc:330] T b4b6cd2b154d4e3aa93d7e5eebf0f761 P 70281fa69b7949aead7a99ec552fa256: stopping tablet replica
I20260807 18:16:03.480370 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 7438
W20260807 18:16:03.508441 7366 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.154.1:44921 (error 108)
I20260807 18:16:03.508819 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 7644
I20260807 18:16:03.513744 8034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:03.512947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.1:44921 (local address 127.0.0.1:58354)
0807 18:16:03.513131 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0807 18:16:03.513490 (+ 359us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.1:44921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260807 18:16:03.514122 7366 catalog_manager.cc:4584] TS 70281fa69b7949aead7a99ec552fa256 (127.18.154.1:44921): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b4b6cd2b154d4e3aa93d7e5eebf0f761: Network error: Client connection negotiation failed: client connection to 127.18.154.1:44921: connect: Connection refused (error 111)
I20260807 18:16:03.528079 8038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:03.509833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60934 (local address 127.18.154.2:45587)
0807 18:16:03.510073 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0807 18:16:03.510081 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:16:03.510109 (+ 28us) server_negotiation.cc:407] Connection header received
0807 18:16:03.510308 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:16:03.510313 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:16:03.510428 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:16:03.510554 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:16:03.512360 (+ 1806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:16:03.513626 (+ 1266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:16:03.515611 (+ 1985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:16:03.516355 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:16:03.516477 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:16:03.520175 (+ 3698us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:16:03.520191 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:16:03.520211 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:16:03.520258 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:16:03.524602 (+ 4344us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:16:03.525490 (+ 888us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:16:03.525500 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:16:03.525504 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:16:03.525866 (+ 362us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:16:03.526324 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:16:03.526329 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:16:03.526333 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:16:03.526819 (+ 486us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:16:03.527170 (+ 351us) server_negotiation.cc:1035] Waiting for connection context
0807 18:16:03.527437 (+ 267us) server_negotiation.cc:299] Negotiation successful
0807 18:16:03.527673 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260807 18:16:03.528869 7825 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:16:03.567823 8034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:03.567252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.1:44921 (local address 127.0.0.1:58360)
0807 18:16:03.567514 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0807 18:16:03.567633 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.1:44921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260807 18:16:03.568950 8034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:03.568534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.1:44921 (local address 127.0.0.1:58366)
0807 18:16:03.568702 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0807 18:16:03.568811 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.1:44921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260807 18:16:03.630016 8034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:03.629306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.1:44921 (local address 127.0.0.1:58372)
0807 18:16:03.629737 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0807 18:16:03.629862 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.1:44921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107}
I20260807 18:16:03.631110 8034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:03.630744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.1:44921 (local address 127.0.0.1:58386)
0807 18:16:03.630889 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0807 18:16:03.630961 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.1:44921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260807 18:16:03.645318 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 7644
I20260807 18:16:03.671324 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 7838
I20260807 18:16:03.685382 8036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:03.672177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54526 (local address 127.18.154.3:44287)
0807 18:16:03.672367 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0807 18:16:03.672372 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:16:03.672392 (+ 20us) server_negotiation.cc:407] Connection header received
0807 18:16:03.672518 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:16:03.672521 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:16:03.672586 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:16:03.672695 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:16:03.674376 (+ 1681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:16:03.675169 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:16:03.676690 (+ 1521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:16:03.677074 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:16:03.677172 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:16:03.679848 (+ 2676us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:16:03.679864 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:16:03.679882 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:16:03.679934 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:16:03.682752 (+ 2818us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:16:03.683449 (+ 697us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:16:03.683456 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:16:03.683458 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:16:03.683599 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:16:03.683998 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:16:03.684002 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:16:03.684003 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:16:03.684418 (+ 415us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:16:03.684708 (+ 290us) server_negotiation.cc:1035] Waiting for connection context
0807 18:16:03.684910 (+ 202us) server_negotiation.cc:299] Negotiation successful
0807 18:16:03.685071 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 18:16:03.686182 8012 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:16:03.711634 8034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:03.711092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.1:44921 (local address 127.0.0.1:58392)
0807 18:16:03.711342 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0807 18:16:03.711454 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.1:44921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260807 18:16:03.712867 8034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:03.712382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.1:44921 (local address 127.0.0.1:58406)
0807 18:16:03.712603 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0807 18:16:03.712703 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.1:44921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110}
I20260807 18:16:03.777163 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 7838
I20260807 18:16:03.801745 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 7339
I20260807 18:16:03.803004 7426 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:16:03.916913 8034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:03.916178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.1:44921 (local address 127.0.0.1:58422)
0807 18:16:03.916447 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0807 18:16:03.916627 (+ 180us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.1:44921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260807 18:16:03.917354 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 7339
I20260807 18:16:03.949008 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.154.62:39535
--webserver_interface=127.18.154.62
--webserver_port=36013
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.62.keytab
--principal=kudu/127.18.154.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.154.20:44041
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.154.62:39535
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:16:04.305199 8086 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:16:04.305491 8086 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:16:04.305666 8086 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:16:04.313166 8086 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:16:04.313270 8086 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:16:04.313374 8086 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:16:04.313457 8086 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:16:04.313529 8086 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:16:04.323684 8086 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.154.20:44041
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.154.62:39535
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.62.keytab
--principal=kudu/127.18.154.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.154.62:39535
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.154.62
--webserver_port=36013
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:16:04.325168 8086 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:16:04.326603 8086 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:16:04.338847 8097 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:16:04.338780 8095 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:16:04.338984 8093 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:16:04.339857 8086 server_base.cc:1034] running on GCE node
I20260807 18:16:04.340731 8086 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:16:04.342274 8086 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:16:04.343487 8086 hybrid_clock.cc:648] HybridClock initialized: now 1786126564343421 us; error 59 us; skew 500 ppm
I20260807 18:16:04.343842 8086 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:16:04 dist-test-slave-gkw7 krb5kdc[6552](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126564, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:16:04.347281 8086 init.cc:377] Logged in from keytab as kudu/127.18.154.62@KRBTEST.COM (short username kudu)
I20260807 18:16:04.348928 8086 webserver.cc:460] Webserver started at http://127.18.154.62:36013/ using document root <none> and password file <none>
I20260807 18:16:04.349687 8086 fs_manager.cc:359] Metadata directory not provided
I20260807 18:16:04.349824 8086 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:16:04.355397 8086 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 18:16:04.358379 8105 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:16:04.359804 8086 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:16:04.360008 8086 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/master-0/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/master-0/wal
uuid: "d800f8a66628417d808b2e4b0961b702"
format_stamp: "Formatted at 2026-08-07 18:15:57 on dist-test-slave-gkw7"
server_key: "9b0188e763314f93735051427425d348"
server_key_iv: "9e7ec036ea217b2c0de08e7ac782e263"
server_key_version: "encryptionkey@0"
I20260807 18:16:04.361112 8086 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:16:04.378400 8086 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:16:04.383179 8086 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:16:04.383484 8086 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:16:04.404188 8086 rpc_server.cc:307] RPC server started. Bound to: 127.18.154.62:39535
I20260807 18:16:04.404207 8177 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.154.62:39535 every 8 connection(s)
I20260807 18:16:04.405900 8086 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
I20260807 18:16:04.407799 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 8086
I20260807 18:16:04.408859 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.154.1:44921
--local_ip_for_outbound_sockets=127.18.154.1
--tserver_master_addrs=127.18.154.62:39535
--webserver_port=36253
--webserver_interface=127.18.154.1
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.1.keytab
--principal=kudu/127.18.154.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.154.20:44041
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 18:16:04.416162 8178 sys_catalog.cc:263] Verifying existing consensus state
I20260807 18:16:04.421414 8178 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d800f8a66628417d808b2e4b0961b702: Bootstrap starting.
I20260807 18:16:04.442761 8178 log.cc:824] T 00000000000000000000000000000000 P d800f8a66628417d808b2e4b0961b702: Log is configured to *not* fsync() on all Append() calls
I20260807 18:16:04.459430 8178 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d800f8a66628417d808b2e4b0961b702: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 18:16:04.460023 8178 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d800f8a66628417d808b2e4b0961b702: Bootstrap complete.
I20260807 18:16:04.469587 8178 raft_consensus.cc:348] T 00000000000000000000000000000000 P d800f8a66628417d808b2e4b0961b702 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d800f8a66628417d808b2e4b0961b702" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 39535 } }
I20260807 18:16:04.470490 8178 raft_consensus.cc:729] T 00000000000000000000000000000000 P d800f8a66628417d808b2e4b0961b702 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: d800f8a66628417d808b2e4b0961b702, State: Initialized, Role: FOLLOWER
I20260807 18:16:04.471158 8178 consensus_queue.cc:260] T 00000000000000000000000000000000 P d800f8a66628417d808b2e4b0961b702 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d800f8a66628417d808b2e4b0961b702" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 39535 } }
I20260807 18:16:04.471401 8178 raft_consensus.cc:388] T 00000000000000000000000000000000 P d800f8a66628417d808b2e4b0961b702 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:16:04.471549 8178 raft_consensus.cc:482] T 00000000000000000000000000000000 P d800f8a66628417d808b2e4b0961b702 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:16:04.471747 8178 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d800f8a66628417d808b2e4b0961b702 [term 2 FOLLOWER]: Advancing to term 3
I20260807 18:16:04.475643 8178 raft_consensus.cc:504] T 00000000000000000000000000000000 P d800f8a66628417d808b2e4b0961b702 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d800f8a66628417d808b2e4b0961b702" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 39535 } }
I20260807 18:16:04.476176 8178 leader_election.cc:302] T 00000000000000000000000000000000 P d800f8a66628417d808b2e4b0961b702 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d800f8a66628417d808b2e4b0961b702; no voters:
I20260807 18:16:04.476858 8178 leader_election.cc:288] T 00000000000000000000000000000000 P d800f8a66628417d808b2e4b0961b702 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260807 18:16:04.477008 8184 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d800f8a66628417d808b2e4b0961b702 [term 3 FOLLOWER]: Leader election won for term 3
I20260807 18:16:04.478166 8184 raft_consensus.cc:686] T 00000000000000000000000000000000 P d800f8a66628417d808b2e4b0961b702 [term 3 LEADER]: Becoming Leader. State: Replica: d800f8a66628417d808b2e4b0961b702, State: Running, Role: LEADER
I20260807 18:16:04.478817 8184 consensus_queue.cc:237] T 00000000000000000000000000000000 P d800f8a66628417d808b2e4b0961b702 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d800f8a66628417d808b2e4b0961b702" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 39535 } }
I20260807 18:16:04.479198 8178 sys_catalog.cc:564] T 00000000000000000000000000000000 P d800f8a66628417d808b2e4b0961b702 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:16:04.481514 8187 sys_catalog.cc:455] T 00000000000000000000000000000000 P d800f8a66628417d808b2e4b0961b702 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d800f8a66628417d808b2e4b0961b702. Latest consensus state: current_term: 3 leader_uuid: "d800f8a66628417d808b2e4b0961b702" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d800f8a66628417d808b2e4b0961b702" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 39535 } } }
I20260807 18:16:04.481632 8186 sys_catalog.cc:455] T 00000000000000000000000000000000 P d800f8a66628417d808b2e4b0961b702 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "d800f8a66628417d808b2e4b0961b702" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d800f8a66628417d808b2e4b0961b702" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 39535 } } }
I20260807 18:16:04.481951 8187 sys_catalog.cc:458] T 00000000000000000000000000000000 P d800f8a66628417d808b2e4b0961b702 [sys.catalog]: This master's current role is: LEADER
I20260807 18:16:04.482028 8186 sys_catalog.cc:458] T 00000000000000000000000000000000 P d800f8a66628417d808b2e4b0961b702 [sys.catalog]: This master's current role is: LEADER
I20260807 18:16:04.483031 8190 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:16:04.502310 8190 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:16:04.504669 8190 catalog_manager.cc:1259] Loaded cluster ID: 67ef141743fc47daa7707f2f49ced5ca
I20260807 18:16:04.504914 8190 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260807 18:16:04.507858 8190 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f86720699ad google::LogMessage::Fail() at ??:0
@ 0x7f867206c88a google::LogMessage::SendToLog() at ??:0
@ 0x7f86720694de google::LogMessage::Flush() at ??:0
@ 0x7f867206aa19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f8687a9c512 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7f8687a99976 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f8687b21610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7f8687b21439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f8674cb5345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f8674cb891c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f8674cb8799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f8674c93eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f8677f6d6db start_thread at ??:0
@ 0x7f8670f0361f clone at ??:0
W20260807 18:16:04.845575 8181 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:16:04.845893 8181 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:16:04.846062 8181 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:16:04.853587 8181 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:16:04.853724 8181 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:16:04.853921 8181 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.154.1
I20260807 18:16:04.864115 8181 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.154.20:44041
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.1.keytab
--principal=kudu/127.18.154.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.154.1:44921
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.154.1
--webserver_port=36253
--tserver_master_addrs=127.18.154.62:39535
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.154.1
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:16:04.865429 8181 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:16:04.866847 8181 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:16:04.880607 8236 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:16:04.880556 8237 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:16:04.880650 8239 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:16:04.881502 8181 server_base.cc:1034] running on GCE node
I20260807 18:16:04.882236 8181 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:16:04.883216 8181 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:16:04.884436 8181 hybrid_clock.cc:648] HybridClock initialized: now 1786126564884350 us; error 54 us; skew 500 ppm
I20260807 18:16:04.884768 8181 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:16:04 dist-test-slave-gkw7 krb5kdc[6552](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126564, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:16:04.888022 8181 init.cc:377] Logged in from keytab as kudu/127.18.154.1@KRBTEST.COM (short username kudu)
I20260807 18:16:04.889798 8181 webserver.cc:460] Webserver started at http://127.18.154.1:36253/ using document root <none> and password file <none>
I20260807 18:16:04.890583 8181 fs_manager.cc:359] Metadata directory not provided
I20260807 18:16:04.890722 8181 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:16:04.896083 8181 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.002s
I20260807 18:16:04.899258 8251 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:16:04.900674 8181 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.001s
I20260807 18:16:04.900846 8181 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/ts-0/wal
uuid: "70281fa69b7949aead7a99ec552fa256"
format_stamp: "Formatted at 2026-08-07 18:15:58 on dist-test-slave-gkw7"
server_key: "e9b0db28c63883e21bb2301158686274"
server_key_iv: "c890fb8f75d03d6d75dcf07022b40aae"
server_key_version: "encryptionkey@0"
I20260807 18:16:04.902011 8181 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:16:04.929153 8181 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:16:04.933723 8181 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:16:04.934033 8181 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:16:04.935078 8181 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:16:04.936969 8259 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260807 18:16:04.943981 8181 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260807 18:16:04.944099 8181 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.008s user 0.000s sys 0.000s
I20260807 18:16:04.944324 8181 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260807 18:16:04.947515 8181 ts_tablet_manager.cc:610] Registered 1 tablets
I20260807 18:16:04.947592 8181 ts_tablet_manager.cc:589] Time spent register tablets: real 0.003s user 0.002s sys 0.000s
I20260807 18:16:04.947995 8259 tablet_bootstrap.cc:492] T b4b6cd2b154d4e3aa93d7e5eebf0f761 P 70281fa69b7949aead7a99ec552fa256: Bootstrap starting.
I20260807 18:16:04.976234 8259 log.cc:824] T b4b6cd2b154d4e3aa93d7e5eebf0f761 P 70281fa69b7949aead7a99ec552fa256: Log is configured to *not* fsync() on all Append() calls
I20260807 18:16:04.989984 8259 tablet_bootstrap.cc:492] T b4b6cd2b154d4e3aa93d7e5eebf0f761 P 70281fa69b7949aead7a99ec552fa256: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 18:16:04.990703 8259 tablet_bootstrap.cc:492] T b4b6cd2b154d4e3aa93d7e5eebf0f761 P 70281fa69b7949aead7a99ec552fa256: Bootstrap complete.
I20260807 18:16:04.990849 8181 rpc_server.cc:307] RPC server started. Bound to: 127.18.154.1:44921
I20260807 18:16:04.990888 8400 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.154.1:44921 every 8 connection(s)
I20260807 18:16:04.992116 8181 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/ts-0/data/info.pb
I20260807 18:16:04.992328 8259 ts_tablet_manager.cc:1397] T b4b6cd2b154d4e3aa93d7e5eebf0f761 P 70281fa69b7949aead7a99ec552fa256: Time spent bootstrapping tablet: real 0.045s user 0.024s sys 0.017s
I20260807 18:16:04.998132 8404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:04.995288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:39535 (local address 127.18.154.1:39337)
0807 18:16:04.996415 (+ 1127us) negotiation.cc:107] Waiting for socket to connect
0807 18:16:04.996758 (+ 343us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.62:39535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":785,"thread_start_us":484,"threads_started":1}
W20260807 18:16:04.998834 8402 heartbeater.cc:644] Failed to heartbeat to 127.18.154.62:39535 (0 consecutive failures): Network error: Failed to ping master at 127.18.154.62:39535: Client connection negotiation failed: client connection to 127.18.154.62:39535: connect: Connection refused (error 111)
I20260807 18:16:04.999805 8404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:04.999390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:39535 (local address 127.18.154.1:54819)
0807 18:16:04.999570 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0807 18:16:04.999662 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.62:39535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260807 18:16:05.000866 8404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:05.000436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:39535 (local address 127.18.154.1:39403)
0807 18:16:05.000606 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0807 18:16:05.000705 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.62:39535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260807 18:16:05.001217 8402 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260807 18:16:05.002550 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 8181
I20260807 18:16:05.001719 8259 raft_consensus.cc:348] T b4b6cd2b154d4e3aa93d7e5eebf0f761 P 70281fa69b7949aead7a99ec552fa256 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fc0d7d72d654b3fbed6817c5715dc59" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 44287 } } peers { permanent_uuid: "70281fa69b7949aead7a99ec552fa256" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 44921 } } peers { permanent_uuid: "20725a0434e54ab9b7d73c8733499c30" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 45587 } }
I20260807 18:16:05.003242 8259 raft_consensus.cc:729] T b4b6cd2b154d4e3aa93d7e5eebf0f761 P 70281fa69b7949aead7a99ec552fa256 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 70281fa69b7949aead7a99ec552fa256, State: Initialized, Role: FOLLOWER
I20260807 18:16:05.003537 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.154.2:45587
--local_ip_for_outbound_sockets=127.18.154.2
--tserver_master_addrs=127.18.154.62:39535
--webserver_port=35589
--webserver_interface=127.18.154.2
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.2.keytab
--principal=kudu/127.18.154.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.154.20:44041
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 18:16:05.003965 8259 consensus_queue.cc:260] T b4b6cd2b154d4e3aa93d7e5eebf0f761 P 70281fa69b7949aead7a99ec552fa256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fc0d7d72d654b3fbed6817c5715dc59" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 44287 } } peers { permanent_uuid: "70281fa69b7949aead7a99ec552fa256" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 44921 } } peers { permanent_uuid: "20725a0434e54ab9b7d73c8733499c30" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 45587 } }
I20260807 18:16:05.005716 8404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:05.005302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:39535 (local address 127.18.154.1:40983)
0807 18:16:05.005457 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0807 18:16:05.005572 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.62:39535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260807 18:16:05.005968 8259 ts_tablet_manager.cc:1428] T b4b6cd2b154d4e3aa93d7e5eebf0f761 P 70281fa69b7949aead7a99ec552fa256: Time spent starting tablet: real 0.014s user 0.011s sys 0.005s
I20260807 18:16:05.006915 8404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:05.006556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:39535 (local address 127.18.154.1:59937)
0807 18:16:05.006704 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0807 18:16:05.006779 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.62:39535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260807 18:16:05.354972 8408 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:16:05.355299 8408 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:16:05.355473 8408 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:16:05.362807 8408 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:16:05.362926 8408 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:16:05.363149 8408 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.154.2
I20260807 18:16:05.372871 8408 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.154.20:44041
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.2.keytab
--principal=kudu/127.18.154.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.154.2:45587
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.154.2
--webserver_port=35589
--tserver_master_addrs=127.18.154.62:39535
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.154.2
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:16:05.374274 8408 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:16:05.375631 8408 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:16:05.389796 8413 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:16:05.389905 8414 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:16:05.389941 8416 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:16:05.390940 8408 server_base.cc:1034] running on GCE node
I20260807 18:16:05.391664 8408 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:16:05.392773 8408 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:16:05.394001 8408 hybrid_clock.cc:648] HybridClock initialized: now 1786126565393923 us; error 73 us; skew 500 ppm
I20260807 18:16:05.394357 8408 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:16:05 dist-test-slave-gkw7 krb5kdc[6552](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126565, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:16:05.397699 8408 init.cc:377] Logged in from keytab as kudu/127.18.154.2@KRBTEST.COM (short username kudu)
I20260807 18:16:05.399240 8408 webserver.cc:460] Webserver started at http://127.18.154.2:35589/ using document root <none> and password file <none>
I20260807 18:16:05.399951 8408 fs_manager.cc:359] Metadata directory not provided
I20260807 18:16:05.400086 8408 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:16:05.405669 8408 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260807 18:16:05.408592 8427 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:16:05.410136 8408 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:16:05.410327 8408 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/ts-1/wal
uuid: "20725a0434e54ab9b7d73c8733499c30"
format_stamp: "Formatted at 2026-08-07 18:15:59 on dist-test-slave-gkw7"
server_key: "7b63ade0bc23bcb7a97748847d6b3f5a"
server_key_iv: "fec9806a8bfee1cbc4c4db5873e15e93"
server_key_version: "encryptionkey@0"
I20260807 18:16:05.411331 8408 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:16:05.428824 8408 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:16:05.433454 8408 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:16:05.433811 8408 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:16:05.435029 8408 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:16:05.436398 8408 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:16:05.436491 8408 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:16:05.436658 8408 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:16:05.436743 8408 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:16:05.470096 8408 rpc_server.cc:307] RPC server started. Bound to: 127.18.154.2:45587
I20260807 18:16:05.470183 8592 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.154.2:45587 every 8 connection(s)
I20260807 18:16:05.471632 8408 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/ts-1/data/info.pb
I20260807 18:16:05.474061 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 8408
I20260807 18:16:05.474694 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.154.3:44287
--local_ip_for_outbound_sockets=127.18.154.3
--tserver_master_addrs=127.18.154.62:39535
--webserver_port=38245
--webserver_interface=127.18.154.3
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.3.keytab
--principal=kudu/127.18.154.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.154.20:44041
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 18:16:05.477061 8596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:05.474978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:39535 (local address 127.18.154.2:59275)
0807 18:16:05.475846 (+ 868us) negotiation.cc:107] Waiting for socket to connect
0807 18:16:05.476123 (+ 277us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.62:39535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":557,"thread_start_us":352,"threads_started":1}
W20260807 18:16:05.477859 8593 heartbeater.cc:644] Failed to heartbeat to 127.18.154.62:39535 (0 consecutive failures): Network error: Failed to ping master at 127.18.154.62:39535: Client connection negotiation failed: client connection to 127.18.154.62:39535: connect: Connection refused (error 111)
I20260807 18:16:05.478945 8596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:05.478443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:39535 (local address 127.18.154.2:50167)
0807 18:16:05.478689 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0807 18:16:05.478793 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.62:39535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260807 18:16:05.479871 8596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:05.479515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:39535 (local address 127.18.154.2:57701)
0807 18:16:05.479672 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0807 18:16:05.479761 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.62:39535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260807 18:16:05.480127 8593 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 18:16:05.849740 8597 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:16:05.850028 8597 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:16:05.850144 8597 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:16:05.857424 8597 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:16:05.857573 8597 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:16:05.857702 8597 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.154.3
I20260807 18:16:05.867431 8597 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.154.20:44041
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.3.keytab
--principal=kudu/127.18.154.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.154.3:44287
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.154.3
--webserver_port=38245
--tserver_master_addrs=127.18.154.62:39535
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.154.3
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:16:05.868726 8597 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:16:05.870111 8597 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:16:05.883543 8603 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:16:05.883664 8602 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:16:05.883667 8608 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:16:05.884516 8597 server_base.cc:1034] running on GCE node
I20260807 18:16:05.885124 8597 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:16:05.886173 8597 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:16:05.887387 8597 hybrid_clock.cc:648] HybridClock initialized: now 1786126565887327 us; error 59 us; skew 500 ppm
I20260807 18:16:05.887720 8597 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:16:05 dist-test-slave-gkw7 krb5kdc[6552](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126565, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:16:05.891160 8597 init.cc:377] Logged in from keytab as kudu/127.18.154.3@KRBTEST.COM (short username kudu)
I20260807 18:16:05.892611 8597 webserver.cc:460] Webserver started at http://127.18.154.3:38245/ using document root <none> and password file <none>
I20260807 18:16:05.893334 8597 fs_manager.cc:359] Metadata directory not provided
I20260807 18:16:05.893467 8597 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:16:05.899130 8597 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.001s sys 0.003s
I20260807 18:16:05.902170 8618 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:16:05.903570 8597 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260807 18:16:05.903762 8597 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/ts-2/wal
uuid: "9fc0d7d72d654b3fbed6817c5715dc59"
format_stamp: "Formatted at 2026-08-07 18:15:59 on dist-test-slave-gkw7"
server_key: "144ea1a0532ead184ac0c8c2989ee2a9"
server_key_iv: "ed37e496730df29551cea2c777ddc950"
server_key_version: "encryptionkey@0"
I20260807 18:16:05.904874 8597 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:16:05.926973 8597 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:16:05.931413 8597 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:16:05.931720 8597 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:16:05.932878 8597 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:16:05.934410 8597 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:16:05.934544 8597 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:16:05.934679 8597 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:16:05.934767 8597 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:16:05.968772 8597 rpc_server.cc:307] RPC server started. Bound to: 127.18.154.3:44287
I20260807 18:16:05.968861 8776 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.154.3:44287 every 8 connection(s)
I20260807 18:16:05.970364 8597 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786126328808105-19048-0/minicluster-data/ts-2/data/info.pb
I20260807 18:16:05.975960 8781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:05.973883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:39535 (local address 127.18.154.3:51631)
0807 18:16:05.974739 (+ 856us) negotiation.cc:107] Waiting for socket to connect
0807 18:16:05.975019 (+ 280us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.62:39535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":564,"thread_start_us":341,"threads_started":1}
W20260807 18:16:05.976859 8779 heartbeater.cc:644] Failed to heartbeat to 127.18.154.62:39535 (0 consecutive failures): Network error: Failed to ping master at 127.18.154.62:39535: Client connection negotiation failed: client connection to 127.18.154.62:39535: connect: Connection refused (error 111)
I20260807 18:16:05.977804 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 8597
I20260807 18:16:05.977862 8781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:05.977379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:39535 (local address 127.18.154.3:34665)
0807 18:16:05.977614 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0807 18:16:05.977716 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.62:39535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260807 18:16:05.978964 8781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:05.978539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:39535 (local address 127.18.154.3:58613)
0807 18:16:05.978721 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0807 18:16:05.978815 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.62:39535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
W20260807 18:16:05.979351 8779 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260807 18:16:06.008451 8783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:06.007707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:39535 (local address 127.18.154.1:49521)
0807 18:16:06.008145 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0807 18:16:06.008236 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.62:39535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":341,"thread_start_us":211,"threads_started":1}
I20260807 18:16:06.481796 8784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:06.480803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:39535 (local address 127.18.154.2:53227)
0807 18:16:06.481351 (+ 548us) negotiation.cc:107] Waiting for socket to connect
0807 18:16:06.481470 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.62:39535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":431,"thread_start_us":285,"threads_started":1}
I20260807 18:16:06.626380 8785 raft_consensus.cc:482] T b4b6cd2b154d4e3aa93d7e5eebf0f761 P 70281fa69b7949aead7a99ec552fa256 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:16:06.626698 8785 raft_consensus.cc:504] T b4b6cd2b154d4e3aa93d7e5eebf0f761 P 70281fa69b7949aead7a99ec552fa256 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fc0d7d72d654b3fbed6817c5715dc59" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 44287 } } peers { permanent_uuid: "70281fa69b7949aead7a99ec552fa256" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 44921 } } peers { permanent_uuid: "20725a0434e54ab9b7d73c8733499c30" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 45587 } }
I20260807 18:16:06.628351 8785 leader_election.cc:288] T b4b6cd2b154d4e3aa93d7e5eebf0f761 P 70281fa69b7949aead7a99ec552fa256 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9fc0d7d72d654b3fbed6817c5715dc59 (127.18.154.3:44287), 20725a0434e54ab9b7d73c8733499c30 (127.18.154.2:45587)
Aug 07 18:16:06 dist-test-slave-gkw7 krb5kdc[6552](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126564, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.1@KRBTEST.COM for kudu/127.18.154.2@KRBTEST.COM
Aug 07 18:16:06 dist-test-slave-gkw7 krb5kdc[6552](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126564, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.1@KRBTEST.COM for kudu/127.18.154.3@KRBTEST.COM
I20260807 18:16:06.655853 8788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:06.629078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.1:48185 (local address 127.18.154.3:44287)
0807 18:16:06.629904 (+ 826us) server_negotiation.cc:206] Beginning negotiation
0807 18:16:06.629924 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0807 18:16:06.630651 (+ 727us) server_negotiation.cc:407] Connection header received
0807 18:16:06.633727 (+ 3076us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:16:06.633742 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:16:06.634312 (+ 570us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:16:06.634736 (+ 424us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:16:06.637855 (+ 3119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:16:06.639831 (+ 1976us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:16:06.642069 (+ 2238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:16:06.642783 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:16:06.643006 (+ 223us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:16:06.647449 (+ 4443us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:16:06.647460 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:16:06.647479 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:16:06.647531 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:16:06.651730 (+ 4199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:16:06.652751 (+ 1021us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:16:06.652760 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:16:06.652773 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:16:06.652939 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:16:06.653380 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:16:06.653385 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:16:06.653388 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:16:06.653996 (+ 608us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:16:06.654207 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0807 18:16:06.655014 (+ 807us) server_negotiation.cc:299] Negotiation successful
0807 18:16:06.655422 (+ 408us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":637,"thread_start_us":502,"threads_started":1}
I20260807 18:16:06.655867 8791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:06.629561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.3:44287 (local address 127.18.154.1:48185)
0807 18:16:06.630435 (+ 874us) negotiation.cc:107] Waiting for socket to connect
0807 18:16:06.630485 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0807 18:16:06.631125 (+ 640us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:16:06.634986 (+ 3861us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:16:06.634995 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:16:06.636262 (+ 1267us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:16:06.637580 (+ 1318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:16:06.637613 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:16:06.640094 (+ 2481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:16:06.640104 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:16:06.641875 (+ 1771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:16:06.641890 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:16:06.643020 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:16:06.643029 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:16:06.643278 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:16:06.643989 (+ 711us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:16:06.644014 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:16:06.647148 (+ 3134us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:16:06.651949 (+ 4801us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:16:06.651953 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:16:06.651956 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:16:06.652592 (+ 636us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:16:06.653056 (+ 464us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:16:06.653062 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:16:06.653066 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:16:06.653259 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:16:06.654149 (+ 890us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:16:06.654157 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:16:06.654508 (+ 351us) client_negotiation.cc:769] Sending connection context
0807 18:16:06.654775 (+ 267us) client_negotiation.cc:240] Negotiation successful
0807 18:16:06.655348 (+ 573us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":766,"mutex_wait_us":238,"spinlock_wait_cycles":49920,"thread_start_us":518,"threads_started":1}
I20260807 18:16:06.655865 8786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:06.629051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.1:53031 (local address 127.18.154.2:45587)
0807 18:16:06.629972 (+ 921us) server_negotiation.cc:206] Beginning negotiation
0807 18:16:06.629993 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0807 18:16:06.630655 (+ 662us) server_negotiation.cc:407] Connection header received
0807 18:16:06.633765 (+ 3110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:16:06.633774 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:16:06.634307 (+ 533us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:16:06.634736 (+ 429us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:16:06.637851 (+ 3115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:16:06.639834 (+ 1983us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:16:06.642018 (+ 2184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:16:06.642664 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:16:06.642871 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:16:06.647127 (+ 4256us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:16:06.647143 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:16:06.647159 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:16:06.647203 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:16:06.651736 (+ 4533us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:16:06.652752 (+ 1016us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:16:06.652759 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:16:06.652766 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:16:06.652935 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:16:06.653400 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:16:06.653405 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:16:06.653407 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:16:06.653997 (+ 590us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:16:06.654225 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0807 18:16:06.655041 (+ 816us) server_negotiation.cc:299] Negotiation successful
0807 18:16:06.655443 (+ 402us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":681,"thread_start_us":766,"threads_started":1}
I20260807 18:16:06.656487 8790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:06.629438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.2:45587 (local address 127.18.154.1:53031)
0807 18:16:06.630435 (+ 997us) negotiation.cc:107] Waiting for socket to connect
0807 18:16:06.630486 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0807 18:16:06.631125 (+ 639us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:16:06.634982 (+ 3857us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:16:06.634995 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:16:06.636262 (+ 1267us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:16:06.637584 (+ 1322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:16:06.637613 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:16:06.640075 (+ 2462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:16:06.640079 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:16:06.641871 (+ 1792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:16:06.641885 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:16:06.642863 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:16:06.642870 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:16:06.643158 (+ 288us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:16:06.644042 (+ 884us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:16:06.644059 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:16:06.646810 (+ 2751us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:16:06.651899 (+ 5089us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:16:06.651912 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:16:06.651927 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:16:06.652591 (+ 664us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:16:06.653059 (+ 468us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:16:06.653064 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:16:06.653067 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:16:06.653270 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:16:06.654152 (+ 882us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:16:06.654163 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:16:06.654508 (+ 345us) client_negotiation.cc:769] Sending connection context
0807 18:16:06.654776 (+ 268us) client_negotiation.cc:240] Negotiation successful
0807 18:16:06.655539 (+ 763us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":836,"mutex_wait_us":538,"spinlock_wait_cycles":23808,"thread_start_us":783,"threads_started":1}
W20260807 18:16:06.659461 8256 leader_election.cc:341] T b4b6cd2b154d4e3aa93d7e5eebf0f761 P 70281fa69b7949aead7a99ec552fa256 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9fc0d7d72d654b3fbed6817c5715dc59 (127.18.154.3:44287): Not found: Tablet not found: b4b6cd2b154d4e3aa93d7e5eebf0f761
W20260807 18:16:06.659817 8255 leader_election.cc:341] T b4b6cd2b154d4e3aa93d7e5eebf0f761 P 70281fa69b7949aead7a99ec552fa256 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 20725a0434e54ab9b7d73c8733499c30 (127.18.154.2:45587): Not found: Tablet not found: b4b6cd2b154d4e3aa93d7e5eebf0f761
I20260807 18:16:06.659976 8255 leader_election.cc:302] T b4b6cd2b154d4e3aa93d7e5eebf0f761 P 70281fa69b7949aead7a99ec552fa256 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70281fa69b7949aead7a99ec552fa256; no voters: 20725a0434e54ab9b7d73c8733499c30, 9fc0d7d72d654b3fbed6817c5715dc59
I20260807 18:16:06.660456 8785 raft_consensus.cc:2726] T b4b6cd2b154d4e3aa93d7e5eebf0f761 P 70281fa69b7949aead7a99ec552fa256 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 18:16:06.981160 8792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:06.980050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:39535 (local address 127.18.154.3:41007)
0807 18:16:06.980741 (+ 691us) negotiation.cc:107] Waiting for socket to connect
0807 18:16:06.980904 (+ 163us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.62:39535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":565,"thread_start_us":411,"threads_started":1}
I20260807 18:16:07.009964 8790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:07.009464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:39535 (local address 127.18.154.1:51203)
0807 18:16:07.009695 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0807 18:16:07.009811 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.62:39535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98}
I20260807 18:16:07.483532 8796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:07.482777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:39535 (local address 127.18.154.2:56441)
0807 18:16:07.483279 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0807 18:16:07.483366 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.62:39535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":384,"thread_start_us":254,"threads_started":1}
I20260807 18:16:07.982865 8798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:07.982175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:39535 (local address 127.18.154.3:51001)
0807 18:16:07.982608 (+ 433us) negotiation.cc:107] Waiting for socket to connect
0807 18:16:07.982694 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.62:39535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":331,"thread_start_us":222,"threads_started":1}
I20260807 18:16:08.011718 8799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:08.010928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:39535 (local address 127.18.154.1:40325)
0807 18:16:08.011446 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0807 18:16:08.011530 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.62:39535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":418,"thread_start_us":249,"threads_started":1}
I20260807 18:16:08.485364 8800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:08.484532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:39535 (local address 127.18.154.2:49119)
0807 18:16:08.485050 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0807 18:16:08.485165 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.62:39535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":410,"thread_start_us":252,"threads_started":1}
I20260807 18:16:08.787248 8801 raft_consensus.cc:482] T b4b6cd2b154d4e3aa93d7e5eebf0f761 P 70281fa69b7949aead7a99ec552fa256 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:16:08.787484 8801 raft_consensus.cc:504] T b4b6cd2b154d4e3aa93d7e5eebf0f761 P 70281fa69b7949aead7a99ec552fa256 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fc0d7d72d654b3fbed6817c5715dc59" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 44287 } } peers { permanent_uuid: "70281fa69b7949aead7a99ec552fa256" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 44921 } } peers { permanent_uuid: "20725a0434e54ab9b7d73c8733499c30" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 45587 } }
I20260807 18:16:08.788334 8801 leader_election.cc:288] T b4b6cd2b154d4e3aa93d7e5eebf0f761 P 70281fa69b7949aead7a99ec552fa256 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9fc0d7d72d654b3fbed6817c5715dc59 (127.18.154.3:44287), 20725a0434e54ab9b7d73c8733499c30 (127.18.154.2:45587)
W20260807 18:16:08.789405 8256 leader_election.cc:341] T b4b6cd2b154d4e3aa93d7e5eebf0f761 P 70281fa69b7949aead7a99ec552fa256 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9fc0d7d72d654b3fbed6817c5715dc59 (127.18.154.3:44287): Not found: Tablet not found: b4b6cd2b154d4e3aa93d7e5eebf0f761
W20260807 18:16:08.789736 8255 leader_election.cc:341] T b4b6cd2b154d4e3aa93d7e5eebf0f761 P 70281fa69b7949aead7a99ec552fa256 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 20725a0434e54ab9b7d73c8733499c30 (127.18.154.2:45587): Not found: Tablet not found: b4b6cd2b154d4e3aa93d7e5eebf0f761
I20260807 18:16:08.789839 8255 leader_election.cc:302] T b4b6cd2b154d4e3aa93d7e5eebf0f761 P 70281fa69b7949aead7a99ec552fa256 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70281fa69b7949aead7a99ec552fa256; no voters: 20725a0434e54ab9b7d73c8733499c30, 9fc0d7d72d654b3fbed6817c5715dc59
I20260807 18:16:08.790167 8801 raft_consensus.cc:2726] T b4b6cd2b154d4e3aa93d7e5eebf0f761 P 70281fa69b7949aead7a99ec552fa256 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 18:16:08.984647 8802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:08.983830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:39535 (local address 127.18.154.3:40219)
0807 18:16:08.984384 (+ 554us) negotiation.cc:107] Waiting for socket to connect
0807 18:16:08.984472 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.62:39535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":434,"thread_start_us":297,"threads_started":1}
I20260807 18:16:09.013746 8804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:09.012783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:39535 (local address 127.18.154.1:39131)
0807 18:16:09.013354 (+ 571us) negotiation.cc:107] Waiting for socket to connect
0807 18:16:09.013477 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.62:39535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":456,"thread_start_us":298,"threads_started":1}
I20260807 18:16:09.487002 8805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:09.486293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:39535 (local address 127.18.154.2:59571)
0807 18:16:09.486735 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0807 18:16:09.486820 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.62:39535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":339,"thread_start_us":219,"threads_started":1}
I20260807 18:16:09.986646 8806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:09.985817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:39535 (local address 127.18.154.3:34157)
0807 18:16:09.986337 (+ 520us) negotiation.cc:107] Waiting for socket to connect
0807 18:16:09.986429 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.62:39535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":402,"thread_start_us":264,"threads_started":1}
I20260807 18:16:10.015566 8809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:10.014853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:39535 (local address 127.18.154.1:48855)
0807 18:16:10.015300 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0807 18:16:10.015388 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.62:39535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":220,"threads_started":1}
I20260807 18:16:10.488754 8810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:10.488021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:39535 (local address 127.18.154.2:60851)
0807 18:16:10.488495 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0807 18:16:10.488580 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.62:39535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":371,"thread_start_us":268,"threads_started":1}
I20260807 18:16:10.592016 8811 raft_consensus.cc:482] T b4b6cd2b154d4e3aa93d7e5eebf0f761 P 70281fa69b7949aead7a99ec552fa256 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:16:10.592244 8811 raft_consensus.cc:504] T b4b6cd2b154d4e3aa93d7e5eebf0f761 P 70281fa69b7949aead7a99ec552fa256 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fc0d7d72d654b3fbed6817c5715dc59" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 44287 } } peers { permanent_uuid: "70281fa69b7949aead7a99ec552fa256" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 44921 } } peers { permanent_uuid: "20725a0434e54ab9b7d73c8733499c30" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 45587 } }
I20260807 18:16:10.593150 8811 leader_election.cc:288] T b4b6cd2b154d4e3aa93d7e5eebf0f761 P 70281fa69b7949aead7a99ec552fa256 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9fc0d7d72d654b3fbed6817c5715dc59 (127.18.154.3:44287), 20725a0434e54ab9b7d73c8733499c30 (127.18.154.2:45587)
W20260807 18:16:10.594063 8256 leader_election.cc:341] T b4b6cd2b154d4e3aa93d7e5eebf0f761 P 70281fa69b7949aead7a99ec552fa256 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9fc0d7d72d654b3fbed6817c5715dc59 (127.18.154.3:44287): Not found: Tablet not found: b4b6cd2b154d4e3aa93d7e5eebf0f761
W20260807 18:16:10.594260 8255 leader_election.cc:341] T b4b6cd2b154d4e3aa93d7e5eebf0f761 P 70281fa69b7949aead7a99ec552fa256 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 20725a0434e54ab9b7d73c8733499c30 (127.18.154.2:45587): Not found: Tablet not found: b4b6cd2b154d4e3aa93d7e5eebf0f761
I20260807 18:16:10.594364 8255 leader_election.cc:302] T b4b6cd2b154d4e3aa93d7e5eebf0f761 P 70281fa69b7949aead7a99ec552fa256 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70281fa69b7949aead7a99ec552fa256; no voters: 20725a0434e54ab9b7d73c8733499c30, 9fc0d7d72d654b3fbed6817c5715dc59
I20260807 18:16:10.594599 8811 raft_consensus.cc:2726] T b4b6cd2b154d4e3aa93d7e5eebf0f761 P 70281fa69b7949aead7a99ec552fa256 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 18:16:10.988746 8813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:10.987709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:39535 (local address 127.18.154.3:48407)
0807 18:16:10.988404 (+ 695us) negotiation.cc:107] Waiting for socket to connect
0807 18:16:10.988519 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.62:39535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":432,"thread_start_us":336,"threads_started":1}
I20260807 18:16:11.017680 8814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:11.016787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:39535 (local address 127.18.154.1:40733)
0807 18:16:11.017312 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0807 18:16:11.017421 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.62:39535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":397,"thread_start_us":233,"threads_started":1}
I20260807 18:16:11.490626 8815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:11.489775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:39535 (local address 127.18.154.2:58029)
0807 18:16:11.490336 (+ 561us) negotiation.cc:107] Waiting for socket to connect
0807 18:16:11.490433 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.62:39535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":432,"thread_start_us":260,"threads_started":1}
I20260807 18:16:11.990751 8816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:11.989870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:39535 (local address 127.18.154.3:38737)
0807 18:16:11.990394 (+ 524us) negotiation.cc:107] Waiting for socket to connect
0807 18:16:11.990485 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.62:39535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":419,"thread_start_us":288,"threads_started":1}
I20260807 18:16:12.020330 8817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:12.018772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:39535 (local address 127.18.154.1:50909)
0807 18:16:12.020008 (+ 1236us) negotiation.cc:107] Waiting for socket to connect
0807 18:16:12.020128 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.62:39535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1110,"thread_start_us":998,"threads_started":1}
I20260807 18:16:12.492810 8818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:12.491852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:39535 (local address 127.18.154.2:44525)
0807 18:16:12.492476 (+ 624us) negotiation.cc:107] Waiting for socket to connect
0807 18:16:12.492595 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.62:39535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":503,"thread_start_us":313,"threads_started":1}
I20260807 18:16:12.992861 8819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:12.992019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:39535 (local address 127.18.154.3:53709)
0807 18:16:12.992570 (+ 551us) negotiation.cc:107] Waiting for socket to connect
0807 18:16:12.992663 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.62:39535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":437,"thread_start_us":255,"threads_started":1}
I20260807 18:16:13.022302 8820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:13.021439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:39535 (local address 127.18.154.1:38173)
0807 18:16:13.022025 (+ 586us) negotiation.cc:107] Waiting for socket to connect
0807 18:16:13.022125 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.62:39535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":469,"thread_start_us":333,"threads_started":1}
I20260807 18:16:13.494709 8821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:13.493948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:39535 (local address 127.18.154.2:52883)
0807 18:16:13.494429 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0807 18:16:13.494519 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.62:39535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":371,"thread_start_us":273,"threads_started":1}
I20260807 18:16:13.994916 8822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:13.994067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:39535 (local address 127.18.154.3:51875)
0807 18:16:13.994543 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0807 18:16:13.994679 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.62:39535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":363,"thread_start_us":250,"threads_started":1}
I20260807 18:16:14.024230 8823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:14.023377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:39535 (local address 127.18.154.1:33231)
0807 18:16:14.023850 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0807 18:16:14.024018 (+ 168us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.62:39535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":240,"threads_started":1}
I20260807 18:16:14.496711 8824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:14.495850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:39535 (local address 127.18.154.2:58257)
0807 18:16:14.496412 (+ 562us) negotiation.cc:107] Waiting for socket to connect
0807 18:16:14.496516 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.62:39535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":433,"thread_start_us":270,"threads_started":1}
I20260807 18:16:14.832439 8825 raft_consensus.cc:482] T b4b6cd2b154d4e3aa93d7e5eebf0f761 P 70281fa69b7949aead7a99ec552fa256 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:16:14.832639 8825 raft_consensus.cc:504] T b4b6cd2b154d4e3aa93d7e5eebf0f761 P 70281fa69b7949aead7a99ec552fa256 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fc0d7d72d654b3fbed6817c5715dc59" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 44287 } } peers { permanent_uuid: "70281fa69b7949aead7a99ec552fa256" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 44921 } } peers { permanent_uuid: "20725a0434e54ab9b7d73c8733499c30" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 45587 } }
I20260807 18:16:14.833570 8825 leader_election.cc:288] T b4b6cd2b154d4e3aa93d7e5eebf0f761 P 70281fa69b7949aead7a99ec552fa256 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9fc0d7d72d654b3fbed6817c5715dc59 (127.18.154.3:44287), 20725a0434e54ab9b7d73c8733499c30 (127.18.154.2:45587)
W20260807 18:16:14.834599 8256 leader_election.cc:341] T b4b6cd2b154d4e3aa93d7e5eebf0f761 P 70281fa69b7949aead7a99ec552fa256 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9fc0d7d72d654b3fbed6817c5715dc59 (127.18.154.3:44287): Not found: Tablet not found: b4b6cd2b154d4e3aa93d7e5eebf0f761
W20260807 18:16:14.834794 8255 leader_election.cc:341] T b4b6cd2b154d4e3aa93d7e5eebf0f761 P 70281fa69b7949aead7a99ec552fa256 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 20725a0434e54ab9b7d73c8733499c30 (127.18.154.2:45587): Not found: Tablet not found: b4b6cd2b154d4e3aa93d7e5eebf0f761
I20260807 18:16:14.834967 8255 leader_election.cc:302] T b4b6cd2b154d4e3aa93d7e5eebf0f761 P 70281fa69b7949aead7a99ec552fa256 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70281fa69b7949aead7a99ec552fa256; no voters: 20725a0434e54ab9b7d73c8733499c30, 9fc0d7d72d654b3fbed6817c5715dc59
I20260807 18:16:14.835284 8825 raft_consensus.cc:2726] T b4b6cd2b154d4e3aa93d7e5eebf0f761 P 70281fa69b7949aead7a99ec552fa256 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 18:16:14.996812 8827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:14.996038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:39535 (local address 127.18.154.3:44509)
0807 18:16:14.996555 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0807 18:16:14.996649 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.62:39535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":402,"thread_start_us":246,"threads_started":1}
I20260807 18:16:15.026850 8828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:15.025522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:39535 (local address 127.18.154.1:46989)
0807 18:16:15.026327 (+ 805us) negotiation.cc:107] Waiting for socket to connect
0807 18:16:15.026507 (+ 180us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.62:39535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":682,"thread_start_us":367,"threads_started":1}
I20260807 18:16:15.498891 8830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:15.498016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:39535 (local address 127.18.154.2:47873)
0807 18:16:15.498590 (+ 574us) negotiation.cc:107] Waiting for socket to connect
0807 18:16:15.498697 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.62:39535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":447,"thread_start_us":324,"threads_started":1}
I20260807 18:16:15.998556 8833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:15.997840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:39535 (local address 127.18.154.3:42507)
0807 18:16:15.998286 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0807 18:16:15.998373 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.62:39535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":342,"thread_start_us":231,"threads_started":1}
I20260807 18:16:16.028656 8834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:16.027897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:39535 (local address 127.18.154.1:49141)
0807 18:16:16.028395 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0807 18:16:16.028478 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.62:39535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":390,"thread_start_us":253,"threads_started":1}
I20260807 18:16:16.500722 8836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:16.499931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:39535 (local address 127.18.154.2:55937)
0807 18:16:16.500400 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0807 18:16:16.500478 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.62:39535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":363,"thread_start_us":233,"threads_started":1}
I20260807 18:16:17.000411 8837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:16.999589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:39535 (local address 127.18.154.3:49655)
0807 18:16:17.000136 (+ 547us) negotiation.cc:107] Waiting for socket to connect
0807 18:16:17.000238 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.62:39535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":417,"thread_start_us":254,"threads_started":1}
I20260807 18:16:17.030627 8838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:17.029815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:39535 (local address 127.18.154.1:45525)
0807 18:16:17.030343 (+ 528us) negotiation.cc:107] Waiting for socket to connect
0807 18:16:17.030436 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.62:39535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":412,"thread_start_us":284,"threads_started":1}
I20260807 18:16:17.502955 8839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:17.501971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:39535 (local address 127.18.154.2:39475)
0807 18:16:17.502614 (+ 643us) negotiation.cc:107] Waiting for socket to connect
0807 18:16:17.502741 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.62:39535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":491,"thread_start_us":335,"threads_started":1}
I20260807 18:16:18.002408 8841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:18.001595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:39535 (local address 127.18.154.3:58585)
0807 18:16:18.002120 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0807 18:16:18.002217 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.62:39535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":403,"thread_start_us":256,"threads_started":1}
I20260807 18:16:18.032722 8842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:18.031775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:39535 (local address 127.18.154.1:36271)
0807 18:16:18.032424 (+ 649us) negotiation.cc:107] Waiting for socket to connect
0807 18:16:18.032516 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.62:39535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":517,"thread_start_us":366,"threads_started":1}
I20260807 18:16:18.505317 8843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:18.504520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:39535 (local address 127.18.154.2:43955)
0807 18:16:18.505033 (+ 513us) negotiation.cc:107] Waiting for socket to connect
0807 18:16:18.505146 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.62:39535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":390,"thread_start_us":262,"threads_started":1}
I20260807 18:16:19.004330 8844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:19.003534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:39535 (local address 127.18.154.3:45723)
0807 18:16:19.004051 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0807 18:16:19.004158 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.62:39535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":397,"thread_start_us":244,"threads_started":1}
I20260807 18:16:19.034739 8845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:19.033920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:39535 (local address 127.18.154.1:51601)
0807 18:16:19.034460 (+ 540us) negotiation.cc:107] Waiting for socket to connect
0807 18:16:19.034549 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.62:39535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":420,"thread_start_us":281,"threads_started":1}
I20260807 18:16:19.507361 8847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:19.506480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:39535 (local address 127.18.154.2:40275)
0807 18:16:19.506955 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0807 18:16:19.507089 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.62:39535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":372,"thread_start_us":238,"threads_started":1}
I20260807 18:16:20.006436 8849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:20.005516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:39535 (local address 127.18.154.3:34823)
0807 18:16:20.006076 (+ 560us) negotiation.cc:107] Waiting for socket to connect
0807 18:16:20.006179 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.62:39535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":409,"thread_start_us":258,"threads_started":1}
I20260807 18:16:20.036763 8850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:20.036025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:39535 (local address 127.18.154.1:40205)
0807 18:16:20.036498 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0807 18:16:20.036585 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.62:39535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":364,"thread_start_us":232,"threads_started":1}
I20260807 18:16:20.509610 8851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:20.508634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:39535 (local address 127.18.154.2:37391)
0807 18:16:20.509265 (+ 631us) negotiation.cc:107] Waiting for socket to connect
0807 18:16:20.509387 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.62:39535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":526,"thread_start_us":268,"threads_started":1}
I20260807 18:16:20.775627 8852 raft_consensus.cc:482] T b4b6cd2b154d4e3aa93d7e5eebf0f761 P 70281fa69b7949aead7a99ec552fa256 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:16:20.775854 8852 raft_consensus.cc:504] T b4b6cd2b154d4e3aa93d7e5eebf0f761 P 70281fa69b7949aead7a99ec552fa256 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fc0d7d72d654b3fbed6817c5715dc59" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 44287 } } peers { permanent_uuid: "70281fa69b7949aead7a99ec552fa256" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 44921 } } peers { permanent_uuid: "20725a0434e54ab9b7d73c8733499c30" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 45587 } }
I20260807 18:16:20.776607 8852 leader_election.cc:288] T b4b6cd2b154d4e3aa93d7e5eebf0f761 P 70281fa69b7949aead7a99ec552fa256 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9fc0d7d72d654b3fbed6817c5715dc59 (127.18.154.3:44287), 20725a0434e54ab9b7d73c8733499c30 (127.18.154.2:45587)
W20260807 18:16:20.777817 8255 leader_election.cc:341] T b4b6cd2b154d4e3aa93d7e5eebf0f761 P 70281fa69b7949aead7a99ec552fa256 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 20725a0434e54ab9b7d73c8733499c30 (127.18.154.2:45587): Not found: Tablet not found: b4b6cd2b154d4e3aa93d7e5eebf0f761
W20260807 18:16:20.778036 8256 leader_election.cc:341] T b4b6cd2b154d4e3aa93d7e5eebf0f761 P 70281fa69b7949aead7a99ec552fa256 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9fc0d7d72d654b3fbed6817c5715dc59 (127.18.154.3:44287): Not found: Tablet not found: b4b6cd2b154d4e3aa93d7e5eebf0f761
I20260807 18:16:20.778225 8256 leader_election.cc:302] T b4b6cd2b154d4e3aa93d7e5eebf0f761 P 70281fa69b7949aead7a99ec552fa256 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70281fa69b7949aead7a99ec552fa256; no voters: 20725a0434e54ab9b7d73c8733499c30, 9fc0d7d72d654b3fbed6817c5715dc59
I20260807 18:16:20.778574 8852 raft_consensus.cc:2726] T b4b6cd2b154d4e3aa93d7e5eebf0f761 P 70281fa69b7949aead7a99ec552fa256 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 18:16:20.980947 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 8181
I20260807 18:16:21.005697 8853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:20.982310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48446 (local address 127.18.154.1:44921)
0807 18:16:20.983101 (+ 791us) server_negotiation.cc:206] Beginning negotiation
0807 18:16:20.983109 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:16:20.983140 (+ 31us) server_negotiation.cc:407] Connection header received
0807 18:16:20.983207 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:16:20.983215 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:16:20.983452 (+ 237us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:16:20.983663 (+ 211us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:16:20.986251 (+ 2588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:16:20.987472 (+ 1221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:16:20.988897 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:16:20.989602 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:16:20.989758 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:16:20.998915 (+ 9157us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:16:20.998932 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:16:20.998951 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:16:20.998987 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:16:21.002430 (+ 3443us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:16:21.003227 (+ 797us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:16:21.003236 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:16:21.003238 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:16:21.003382 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:16:21.003832 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:16:21.003838 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:16:21.003841 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:16:21.004254 (+ 413us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:16:21.004584 (+ 330us) server_negotiation.cc:1035] Waiting for connection context
0807 18:16:21.004947 (+ 363us) server_negotiation.cc:299] Negotiation successful
0807 18:16:21.005283 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":549,"thread_start_us":354,"threads_started":1}
I20260807 18:16:21.007436 8395 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:16:21.008883 8855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:21.007788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:39535 (local address 127.18.154.3:36075)
0807 18:16:21.008411 (+ 623us) negotiation.cc:107] Waiting for socket to connect
0807 18:16:21.008621 (+ 210us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.62:39535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":458,"thread_start_us":276,"threads_started":1}
I20260807 18:16:21.099712 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 8181
I20260807 18:16:21.099773 8857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:21.098614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:39535 (local address 127.18.154.1:57201)
0807 18:16:21.099262 (+ 648us) negotiation.cc:107] Waiting for socket to connect
0807 18:16:21.099442 (+ 180us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.62:39535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":502,"thread_start_us":320,"threads_started":1}
I20260807 18:16:21.129599 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 8408
I20260807 18:16:21.144260 8858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:21.130506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52380 (local address 127.18.154.2:45587)
0807 18:16:21.131165 (+ 659us) server_negotiation.cc:206] Beginning negotiation
0807 18:16:21.131170 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:16:21.131189 (+ 19us) server_negotiation.cc:407] Connection header received
0807 18:16:21.131250 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:16:21.131254 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:16:21.131322 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:16:21.131425 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:16:21.132845 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:16:21.133921 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:16:21.135162 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:16:21.135604 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:16:21.135706 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:16:21.138640 (+ 2934us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:16:21.138645 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:16:21.138648 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:16:21.138682 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:16:21.141293 (+ 2611us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:16:21.142235 (+ 942us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:16:21.142240 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:16:21.142242 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:16:21.142364 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:16:21.142842 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:16:21.142846 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:16:21.142848 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:16:21.143116 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:16:21.143335 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0807 18:16:21.143618 (+ 283us) server_negotiation.cc:299] Negotiation successful
0807 18:16:21.143807 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":518,"thread_start_us":354,"threads_started":1}
I20260807 18:16:21.145087 8587 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:16:21.229977 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 8408
I20260807 18:16:21.253304 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 8597
I20260807 18:16:21.268483 8863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:21.254286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36508 (local address 127.18.154.3:44287)
0807 18:16:21.254985 (+ 699us) server_negotiation.cc:206] Beginning negotiation
0807 18:16:21.254991 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:16:21.255011 (+ 20us) server_negotiation.cc:407] Connection header received
0807 18:16:21.255087 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:16:21.255092 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:16:21.255168 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:16:21.255269 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:16:21.256842 (+ 1573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:16:21.257879 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:16:21.259146 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:16:21.259571 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:16:21.259672 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:16:21.262686 (+ 3014us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:16:21.262691 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:16:21.262695 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:16:21.262731 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:16:21.265613 (+ 2882us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:16:21.266329 (+ 716us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:16:21.266334 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:16:21.266337 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:16:21.266526 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:16:21.266996 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:16:21.267001 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:16:21.267003 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:16:21.267331 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:16:21.267567 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0807 18:16:21.267841 (+ 274us) server_negotiation.cc:299] Negotiation successful
0807 18:16:21.268013 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":562,"thread_start_us":383,"threads_started":1}
I20260807 18:16:21.269577 8770 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:16:21.357020 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 8597
2026-08-07T18:16:21Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (25401 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:16:21 dist-test-slave-gkw7 krb5kdc[8868](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:16:21 dist-test-slave-gkw7 krb5kdc[8868](info): set up 2 sockets
Aug 07 18:16:21 dist-test-slave-gkw7 krb5kdc[8868](info): commencing operation
krb5kdc: starting...
W20260807 18:16:22.673141 19048 mini_kdc.cc:121] Time spent starting KDC: real 1.264s user 0.003s sys 0.035s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/krb5kdc/test-user.keytab.
Aug 07 18:16:22 dist-test-slave-gkw7 krb5kdc[8868](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126582, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:16:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:16:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.154.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.154.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.154.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.62.keytab.
Entry for principal kudu/127.18.154.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.62.keytab.
I20260807 18:16:22.973752 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.154.62:46089
--webserver_interface=127.18.154.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.62.keytab
--principal=kudu/127.18.154.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.154.20:44143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.154.62:46089
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:16:23.334496 8885 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:16:23.334815 8885 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:16:23.334995 8885 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:16:23.342537 8885 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260807 18:16:23.342648 8885 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260807 18:16:23.342687 8885 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260807 18:16:23.342784 8885 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:16:23.342875 8885 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:16:23.342995 8885 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:16:23.343052 8885 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:16:23.343132 8885 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:16:23.354074 8885 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.154.20:44143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.18.154.62:46089
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.62.keytab
--principal=kudu/127.18.154.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.154.62:46089
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.154.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:16:23.355584 8885 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:16:23.357182 8885 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:16:23.370829 8896 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:16:23.370962 8897 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:16:23.370987 8899 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:16:23.372021 8885 server_base.cc:1034] running on GCE node
I20260807 18:16:23.372957 8885 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:16:23.374522 8885 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:16:23.375782 8885 hybrid_clock.cc:648] HybridClock initialized: now 1786126583375694 us; error 63 us; skew 500 ppm
I20260807 18:16:23.376142 8885 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:16:23 dist-test-slave-gkw7 krb5kdc[8868](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126583, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:16:23.379527 8885 init.cc:377] Logged in from keytab as kudu/127.18.154.62@KRBTEST.COM (short username kudu)
I20260807 18:16:23.381106 8885 webserver.cc:460] Webserver started at http://127.18.154.62:44395/ using document root <none> and password file <none>
I20260807 18:16:23.381961 8885 fs_manager.cc:359] Metadata directory not provided
I20260807 18:16:23.382102 8885 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:16:23.382445 8885 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:16:23.385617 8885 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/master-0/data/instance:
uuid: "74b7e49ff11046dba46d21f00d9fd15a"
format_stamp: "Formatted at 2026-08-07 18:16:23 on dist-test-slave-gkw7"
server_key: "25521df0306039dcaea0a5175ac320c4"
server_key_iv: "3c597486b4bad2870224c9e0ab5dee35"
server_key_version: "encryptionkey@0"
I20260807 18:16:23.386436 8885 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/master-0/wal/instance:
uuid: "74b7e49ff11046dba46d21f00d9fd15a"
format_stamp: "Formatted at 2026-08-07 18:16:23 on dist-test-slave-gkw7"
server_key: "25521df0306039dcaea0a5175ac320c4"
server_key_iv: "3c597486b4bad2870224c9e0ab5dee35"
server_key_version: "encryptionkey@0"
I20260807 18:16:23.391848 8885 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 18:16:23.395534 8910 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:16:23.396952 8885 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 18:16:23.397158 8885 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/master-0/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/master-0/wal
uuid: "74b7e49ff11046dba46d21f00d9fd15a"
format_stamp: "Formatted at 2026-08-07 18:16:23 on dist-test-slave-gkw7"
server_key: "25521df0306039dcaea0a5175ac320c4"
server_key_iv: "3c597486b4bad2870224c9e0ab5dee35"
server_key_version: "encryptionkey@0"
I20260807 18:16:23.397392 8885 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:16:23.419800 8885 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:16:23.424711 8885 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:16:23.424980 8885 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:16:23.445169 8885 rpc_server.cc:307] RPC server started. Bound to: 127.18.154.62:46089
I20260807 18:16:23.445194 8977 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.154.62:46089 every 8 connection(s)
I20260807 18:16:23.446699 8885 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
I20260807 18:16:23.450904 8978 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:16:23.455214 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 8885
I20260807 18:16:23.455457 19048 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/master-0/wal/instance
I20260807 18:16:23.456391 19048 external_mini_cluster.cc:1428] Setting key 0f7837da1a4a13f6848a8f3d70e90aee
I20260807 18:16:23.463802 8978 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 74b7e49ff11046dba46d21f00d9fd15a: Bootstrap starting.
I20260807 18:16:23.467833 8978 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 74b7e49ff11046dba46d21f00d9fd15a: Neither blocks nor log segments found. Creating new log.
I20260807 18:16:23.469117 8978 log.cc:824] T 00000000000000000000000000000000 P 74b7e49ff11046dba46d21f00d9fd15a: Log is configured to *not* fsync() on all Append() calls
I20260807 18:16:23.472970 8978 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 74b7e49ff11046dba46d21f00d9fd15a: No bootstrap required, opened a new log
Aug 07 18:16:23 dist-test-slave-gkw7 krb5kdc[8868](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126582, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.154.62@KRBTEST.COM
I20260807 18:16:23.480958 8978 raft_consensus.cc:348] T 00000000000000000000000000000000 P 74b7e49ff11046dba46d21f00d9fd15a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74b7e49ff11046dba46d21f00d9fd15a" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 46089 } }
I20260807 18:16:23.481369 8978 raft_consensus.cc:374] T 00000000000000000000000000000000 P 74b7e49ff11046dba46d21f00d9fd15a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:16:23.481508 8978 raft_consensus.cc:729] T 00000000000000000000000000000000 P 74b7e49ff11046dba46d21f00d9fd15a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74b7e49ff11046dba46d21f00d9fd15a, State: Initialized, Role: FOLLOWER
I20260807 18:16:23.482179 8978 consensus_queue.cc:260] T 00000000000000000000000000000000 P 74b7e49ff11046dba46d21f00d9fd15a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74b7e49ff11046dba46d21f00d9fd15a" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 46089 } }
I20260807 18:16:23.482457 8978 raft_consensus.cc:388] T 00000000000000000000000000000000 P 74b7e49ff11046dba46d21f00d9fd15a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:16:23.482621 8978 raft_consensus.cc:482] T 00000000000000000000000000000000 P 74b7e49ff11046dba46d21f00d9fd15a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:16:23.482847 8978 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 74b7e49ff11046dba46d21f00d9fd15a [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:16:23.483076 8982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:23.458804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59168 (local address 127.18.154.62:46089)
0807 18:16:23.459628 (+ 824us) server_negotiation.cc:206] Beginning negotiation
0807 18:16:23.459642 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 18:16:23.459686 (+ 44us) server_negotiation.cc:407] Connection header received
0807 18:16:23.461959 (+ 2273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:16:23.461988 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:16:23.462454 (+ 466us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:16:23.462810 (+ 356us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:16:23.464954 (+ 2144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:16:23.466598 (+ 1644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:16:23.468654 (+ 2056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:16:23.469475 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:16:23.469757 (+ 282us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:16:23.474931 (+ 5174us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:16:23.474942 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:16:23.474957 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:16:23.474997 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:16:23.478333 (+ 3336us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:16:23.479147 (+ 814us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:16:23.479156 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:16:23.479163 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:16:23.479361 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:16:23.479784 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:16:23.479790 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:16:23.479793 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:16:23.480337 (+ 544us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:16:23.480660 (+ 323us) server_negotiation.cc:1035] Waiting for connection context
0807 18:16:23.481139 (+ 479us) server_negotiation.cc:299] Negotiation successful
0807 18:16:23.481709 (+ 570us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":603,"thread_start_us":376,"threads_started":1}
I20260807 18:16:23.485872 8978 raft_consensus.cc:504] T 00000000000000000000000000000000 P 74b7e49ff11046dba46d21f00d9fd15a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74b7e49ff11046dba46d21f00d9fd15a" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 46089 } }
I20260807 18:16:23.486490 8978 leader_election.cc:302] T 00000000000000000000000000000000 P 74b7e49ff11046dba46d21f00d9fd15a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 74b7e49ff11046dba46d21f00d9fd15a; no voters:
I20260807 18:16:23.487099 8978 leader_election.cc:288] T 00000000000000000000000000000000 P 74b7e49ff11046dba46d21f00d9fd15a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:16:23.487214 8984 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 74b7e49ff11046dba46d21f00d9fd15a [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:16:23.488618 8984 raft_consensus.cc:686] T 00000000000000000000000000000000 P 74b7e49ff11046dba46d21f00d9fd15a [term 1 LEADER]: Becoming Leader. State: Replica: 74b7e49ff11046dba46d21f00d9fd15a, State: Running, Role: LEADER
I20260807 18:16:23.489176 8984 consensus_queue.cc:237] T 00000000000000000000000000000000 P 74b7e49ff11046dba46d21f00d9fd15a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74b7e49ff11046dba46d21f00d9fd15a" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 46089 } }
I20260807 18:16:23.489454 8978 sys_catalog.cc:564] T 00000000000000000000000000000000 P 74b7e49ff11046dba46d21f00d9fd15a [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:16:23.492333 8986 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74b7e49ff11046dba46d21f00d9fd15a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 74b7e49ff11046dba46d21f00d9fd15a. Latest consensus state: current_term: 1 leader_uuid: "74b7e49ff11046dba46d21f00d9fd15a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74b7e49ff11046dba46d21f00d9fd15a" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 46089 } } }
I20260807 18:16:23.492807 8986 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74b7e49ff11046dba46d21f00d9fd15a [sys.catalog]: This master's current role is: LEADER
I20260807 18:16:23.494338 8985 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74b7e49ff11046dba46d21f00d9fd15a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "74b7e49ff11046dba46d21f00d9fd15a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74b7e49ff11046dba46d21f00d9fd15a" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 46089 } } }
I20260807 18:16:23.494720 8985 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74b7e49ff11046dba46d21f00d9fd15a [sys.catalog]: This master's current role is: LEADER
I20260807 18:16:23.495206 8995 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:16:23.500988 8995 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:16:23.512287 8995 catalog_manager.cc:1347] Generated new cluster ID: 64638edcd2e6457cb96fce940f3801a9
I20260807 18:16:23.512424 8995 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:16:23.535847 8995 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:16:23.537283 8995 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:16:23.545775 8995 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 74b7e49ff11046dba46d21f00d9fd15a: Generated new TSK 0
I20260807 18:16:23.546913 8995 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.154.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.154.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.154.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.1.keytab.
Entry for principal kudu/127.18.154.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.1.keytab.
I20260807 18:16:23.625279 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.154.1:0
--local_ip_for_outbound_sockets=127.18.154.1
--webserver_interface=127.18.154.1
--webserver_port=0
--tserver_master_addrs=127.18.154.62:46089
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.1.keytab
--principal=kudu/127.18.154.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.154.20:44143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:16:23.982909 9011 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:16:23.983217 9011 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:16:23.983378 9011 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:16:23.991097 9011 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:16:23.991221 9011 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:16:23.991338 9011 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.154.1
I20260807 18:16:24.002064 9011 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.154.20:44143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.1.keytab
--principal=kudu/127.18.154.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.154.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.154.1
--webserver_port=0
--tserver_master_addrs=127.18.154.62:46089
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.154.1
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:16:24.003495 9011 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:16:24.004930 9011 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:16:24.019001 9018 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:16:24.019057 9020 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:16:24.019189 9022 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:16:24.020435 9011 server_base.cc:1034] running on GCE node
I20260807 18:16:24.021054 9011 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:16:24.022112 9011 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:16:24.023371 9011 hybrid_clock.cc:648] HybridClock initialized: now 1786126584023282 us; error 62 us; skew 500 ppm
I20260807 18:16:24.023784 9011 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:16:24 dist-test-slave-gkw7 krb5kdc[8868](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126584, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:16:24.027638 9011 init.cc:377] Logged in from keytab as kudu/127.18.154.1@KRBTEST.COM (short username kudu)
I20260807 18:16:24.029224 9011 webserver.cc:460] Webserver started at http://127.18.154.1:40111/ using document root <none> and password file <none>
I20260807 18:16:24.030043 9011 fs_manager.cc:359] Metadata directory not provided
I20260807 18:16:24.030169 9011 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:16:24.030468 9011 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:16:24.033812 9011 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/ts-0/data/instance:
uuid: "b3f50a9f91964b79b66d73c38ac9ab47"
format_stamp: "Formatted at 2026-08-07 18:16:24 on dist-test-slave-gkw7"
server_key: "9bcaaa5a6d0b685de9523261f10c7145"
server_key_iv: "6e711d227088d40fdf26e493e31bed16"
server_key_version: "encryptionkey@0"
I20260807 18:16:24.034694 9011 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/ts-0/wal/instance:
uuid: "b3f50a9f91964b79b66d73c38ac9ab47"
format_stamp: "Formatted at 2026-08-07 18:16:24 on dist-test-slave-gkw7"
server_key: "9bcaaa5a6d0b685de9523261f10c7145"
server_key_iv: "6e711d227088d40fdf26e493e31bed16"
server_key_version: "encryptionkey@0"
I20260807 18:16:24.040458 9011 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 18:16:24.044592 9034 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:16:24.046324 9011 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 18:16:24.046480 9011 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/ts-0/wal
uuid: "b3f50a9f91964b79b66d73c38ac9ab47"
format_stamp: "Formatted at 2026-08-07 18:16:24 on dist-test-slave-gkw7"
server_key: "9bcaaa5a6d0b685de9523261f10c7145"
server_key_iv: "6e711d227088d40fdf26e493e31bed16"
server_key_version: "encryptionkey@0"
I20260807 18:16:24.046679 9011 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:16:24.067013 9011 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:16:24.072184 9011 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:16:24.072489 9011 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:16:24.073586 9011 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:16:24.075232 9011 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:16:24.075316 9011 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:16:24.075416 9011 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:16:24.075466 9011 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:16:24.114333 9011 rpc_server.cc:307] RPC server started. Bound to: 127.18.154.1:34909
I20260807 18:16:24.114454 9192 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.154.1:34909 every 8 connection(s)
I20260807 18:16:24.115959 9011 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/ts-0/data/info.pb
I20260807 18:16:24.116390 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 9011
I20260807 18:16:24.116668 19048 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/ts-0/wal/instance
I20260807 18:16:24.117960 19048 external_mini_cluster.cc:1428] Setting key b1e0807047214277c378184bdb265b6f
Aug 07 18:16:24 dist-test-slave-gkw7 krb5kdc[8868](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126584, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.1@KRBTEST.COM for kudu/127.18.154.62@KRBTEST.COM
I20260807 18:16:24.150724 9195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:24.120223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.1:37799 (local address 127.18.154.62:46089)
0807 18:16:24.129978 (+ 9755us) server_negotiation.cc:206] Beginning negotiation
0807 18:16:24.129983 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:16:24.130003 (+ 20us) server_negotiation.cc:407] Connection header received
0807 18:16:24.130060 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:16:24.130064 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:16:24.130136 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:16:24.130252 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:16:24.132206 (+ 1954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:16:24.133447 (+ 1241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:16:24.136351 (+ 2904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:16:24.137232 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:16:24.137404 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:16:24.142206 (+ 4802us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:16:24.142216 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:16:24.142225 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:16:24.142281 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:16:24.146529 (+ 4248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:16:24.147517 (+ 988us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:16:24.147527 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:16:24.147540 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:16:24.147739 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:16:24.148287 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:16:24.148296 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:16:24.148302 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:16:24.148760 (+ 458us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:16:24.148929 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0807 18:16:24.149779 (+ 850us) server_negotiation.cc:299] Negotiation successful
0807 18:16:24.150085 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9619,"thread_start_us":9473,"threads_started":1}
I20260807 18:16:24.151707 9196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:24.121065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:46089 (local address 127.18.154.1:37799)
0807 18:16:24.122258 (+ 1193us) negotiation.cc:107] Waiting for socket to connect
0807 18:16:24.122329 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0807 18:16:24.125737 (+ 3408us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:16:24.130587 (+ 4850us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:16:24.130599 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:16:24.131283 (+ 684us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:16:24.131971 (+ 688us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:16:24.131991 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:16:24.133707 (+ 1716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:16:24.133716 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:16:24.136180 (+ 2464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:16:24.136196 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:16:24.137507 (+ 1311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:16:24.137513 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:16:24.137858 (+ 345us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:16:24.138672 (+ 814us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:16:24.138698 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:16:24.141913 (+ 3215us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:16:24.146721 (+ 4808us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:16:24.146729 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:16:24.146746 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:16:24.147362 (+ 616us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:16:24.147885 (+ 523us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:16:24.147889 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:16:24.147891 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:16:24.148143 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:16:24.148905 (+ 762us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:16:24.148911 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:16:24.149313 (+ 402us) client_negotiation.cc:769] Sending connection context
0807 18:16:24.149644 (+ 331us) client_negotiation.cc:240] Negotiation successful
0807 18:16:24.150082 (+ 438us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":759,"thread_start_us":445,"threads_started":1}
I20260807 18:16:24.153813 9193 heartbeater.cc:347] Connected to a master server at 127.18.154.62:46089
I20260807 18:16:24.154204 9193 heartbeater.cc:464] Registering TS with master...
I20260807 18:16:24.155066 9193 heartbeater.cc:511] Master 127.18.154.62:46089 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.18.154.2@KRBTEST.COM; defaulting to no policy
I20260807 18:16:24.157667 8930 ts_manager.cc:194] Registered new tserver with Master: b3f50a9f91964b79b66d73c38ac9ab47 (127.18.154.1:34909)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.154.2@KRBTEST.COM" created.
I20260807 18:16:24.159412 8930 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.154.1@KRBTEST.COM'} at 127.18.154.1:37799
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.154.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.2.keytab.
Entry for principal kudu/127.18.154.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.2.keytab.
I20260807 18:16:24.194674 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.154.2:0
--local_ip_for_outbound_sockets=127.18.154.2
--webserver_interface=127.18.154.2
--webserver_port=0
--tserver_master_addrs=127.18.154.62:46089
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.2.keytab
--principal=kudu/127.18.154.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.154.20:44143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:16:24.546806 9199 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:16:24.547139 9199 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:16:24.547314 9199 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:16:24.554880 9199 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:16:24.555037 9199 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:16:24.555230 9199 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.154.2
I20260807 18:16:24.564924 9199 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.154.20:44143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.2.keytab
--principal=kudu/127.18.154.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.154.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.154.2
--webserver_port=0
--tserver_master_addrs=127.18.154.62:46089
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.154.2
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:16:24.566303 9199 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:16:24.567695 9199 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:16:24.581565 9204 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:16:24.581599 9205 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:16:24.581687 9210 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:16:24.583137 9199 server_base.cc:1034] running on GCE node
I20260807 18:16:24.583735 9199 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:16:24.584651 9199 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:16:24.585846 9199 hybrid_clock.cc:648] HybridClock initialized: now 1786126584585791 us; error 45 us; skew 500 ppm
I20260807 18:16:24.586141 9199 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:16:24 dist-test-slave-gkw7 krb5kdc[8868](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126584, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:16:24.589350 9199 init.cc:377] Logged in from keytab as kudu/127.18.154.2@KRBTEST.COM (short username kudu)
I20260807 18:16:24.591048 9199 webserver.cc:460] Webserver started at http://127.18.154.2:44051/ using document root <none> and password file <none>
I20260807 18:16:24.591728 9199 fs_manager.cc:359] Metadata directory not provided
I20260807 18:16:24.591830 9199 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:16:24.592092 9199 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:16:24.595130 9199 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/ts-1/data/instance:
uuid: "3487895bdebd4925b1e0ee7b961c18d9"
format_stamp: "Formatted at 2026-08-07 18:16:24 on dist-test-slave-gkw7"
server_key: "5479e736d511ce1d944ebffc9f018b16"
server_key_iv: "3268433d2eac4524a60aef8bd486c087"
server_key_version: "encryptionkey@0"
I20260807 18:16:24.595873 9199 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/ts-1/wal/instance:
uuid: "3487895bdebd4925b1e0ee7b961c18d9"
format_stamp: "Formatted at 2026-08-07 18:16:24 on dist-test-slave-gkw7"
server_key: "5479e736d511ce1d944ebffc9f018b16"
server_key_iv: "3268433d2eac4524a60aef8bd486c087"
server_key_version: "encryptionkey@0"
I20260807 18:16:24.601183 9199 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.003s
I20260807 18:16:24.604924 9218 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:16:24.606348 9199 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:16:24.606509 9199 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/ts-1/wal
uuid: "3487895bdebd4925b1e0ee7b961c18d9"
format_stamp: "Formatted at 2026-08-07 18:16:24 on dist-test-slave-gkw7"
server_key: "5479e736d511ce1d944ebffc9f018b16"
server_key_iv: "3268433d2eac4524a60aef8bd486c087"
server_key_version: "encryptionkey@0"
I20260807 18:16:24.606725 9199 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:16:24.632455 9199 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:16:24.636498 9199 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:16:24.636746 9199 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:16:24.637737 9199 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:16:24.639086 9199 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:16:24.639173 9199 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:16:24.639258 9199 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:16:24.639302 9199 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:16:24.674343 9199 rpc_server.cc:307] RPC server started. Bound to: 127.18.154.2:42929
I20260807 18:16:24.674396 9373 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.154.2:42929 every 8 connection(s)
I20260807 18:16:24.675689 9199 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/ts-1/data/info.pb
I20260807 18:16:24.678064 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 9199
I20260807 18:16:24.678292 19048 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/ts-1/wal/instance
I20260807 18:16:24.678916 19048 external_mini_cluster.cc:1428] Setting key 7e53cd1cff3be437be6495d6b52ba13c
Aug 07 18:16:24 dist-test-slave-gkw7 krb5kdc[8868](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126584, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.2@KRBTEST.COM for kudu/127.18.154.62@KRBTEST.COM
I20260807 18:16:24.697768 9381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:24.678439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.2:46457 (local address 127.18.154.62:46089)
0807 18:16:24.678944 (+ 505us) server_negotiation.cc:206] Beginning negotiation
0807 18:16:24.678951 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:16:24.680108 (+ 1157us) server_negotiation.cc:407] Connection header received
0807 18:16:24.682193 (+ 2085us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:16:24.682202 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:16:24.682295 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:16:24.682416 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:16:24.683901 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:16:24.685060 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:16:24.686653 (+ 1593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:16:24.687398 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:16:24.687546 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:16:24.691067 (+ 3521us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:16:24.691076 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:16:24.691084 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:16:24.691154 (+ 70us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:16:24.694202 (+ 3048us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:16:24.695194 (+ 992us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:16:24.695198 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:16:24.695201 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:16:24.695324 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:16:24.695831 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:16:24.695834 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:16:24.695836 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:16:24.696095 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:16:24.696257 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0807 18:16:24.697250 (+ 993us) server_negotiation.cc:299] Negotiation successful
0807 18:16:24.697423 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":363,"thread_start_us":234,"threads_started":1}
I20260807 18:16:24.699313 9382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:24.679046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:46089 (local address 127.18.154.2:46457)
0807 18:16:24.679934 (+ 888us) negotiation.cc:107] Waiting for socket to connect
0807 18:16:24.679989 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0807 18:16:24.681920 (+ 1931us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:16:24.682610 (+ 690us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:16:24.682618 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:16:24.683141 (+ 523us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:16:24.683722 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:16:24.683740 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:16:24.685213 (+ 1473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:16:24.685216 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:16:24.686506 (+ 1290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:16:24.686518 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:16:24.687602 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:16:24.687606 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:16:24.687811 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:16:24.688513 (+ 702us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:16:24.688536 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:16:24.690847 (+ 2311us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:16:24.694344 (+ 3497us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:16:24.694355 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:16:24.694376 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:16:24.695034 (+ 658us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:16:24.695439 (+ 405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:16:24.695446 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:16:24.695452 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:16:24.695715 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:16:24.696267 (+ 552us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:16:24.696279 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:16:24.696817 (+ 538us) client_negotiation.cc:769] Sending connection context
0807 18:16:24.697133 (+ 316us) client_negotiation.cc:240] Negotiation successful
0807 18:16:24.697804 (+ 671us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":609,"thread_start_us":357,"threads_started":1}
I20260807 18:16:24.700874 9376 heartbeater.cc:347] Connected to a master server at 127.18.154.62:46089
I20260807 18:16:24.701243 9376 heartbeater.cc:464] Registering TS with master...
I20260807 18:16:24.702250 9376 heartbeater.cc:511] Master 127.18.154.62:46089 requested a full tablet report, sending...
I20260807 18:16:24.703924 8930 ts_manager.cc:194] Registered new tserver with Master: 3487895bdebd4925b1e0ee7b961c18d9 (127.18.154.2:42929)
I20260807 18:16:24.705240 8930 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.154.2@KRBTEST.COM'} at 127.18.154.2:46457
WARNING: no policy specified for kudu/127.18.154.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.154.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.154.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.3.keytab.
Entry for principal kudu/127.18.154.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.3.keytab.
I20260807 18:16:24.748538 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.154.3:0
--local_ip_for_outbound_sockets=127.18.154.3
--webserver_interface=127.18.154.3
--webserver_port=0
--tserver_master_addrs=127.18.154.62:46089
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.3.keytab
--principal=kudu/127.18.154.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.154.20:44143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:16:25.096572 9388 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:16:25.096855 9388 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:16:25.096952 9388 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:16:25.104386 9388 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:16:25.104501 9388 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:16:25.104614 9388 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.154.3
I20260807 18:16:25.114845 9388 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.154.20:44143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.3.keytab
--principal=kudu/127.18.154.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.154.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.154.3
--webserver_port=0
--tserver_master_addrs=127.18.154.62:46089
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.154.3
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:16:25.116139 9388 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:16:25.117590 9388 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:16:25.131821 9393 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:16:25.131760 9396 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:16:25.131785 9394 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:16:25.132820 9388 server_base.cc:1034] running on GCE node
I20260807 18:16:25.133493 9388 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:16:25.134617 9388 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:16:25.135888 9388 hybrid_clock.cc:648] HybridClock initialized: now 1786126585135839 us; error 59 us; skew 500 ppm
I20260807 18:16:25.136248 9388 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:16:25 dist-test-slave-gkw7 krb5kdc[8868](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126585, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:16:25.139799 9388 init.cc:377] Logged in from keytab as kudu/127.18.154.3@KRBTEST.COM (short username kudu)
I20260807 18:16:25.141451 9388 webserver.cc:460] Webserver started at http://127.18.154.3:43187/ using document root <none> and password file <none>
I20260807 18:16:25.142285 9388 fs_manager.cc:359] Metadata directory not provided
I20260807 18:16:25.142452 9388 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:16:25.142822 9388 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:16:25.146227 9388 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/ts-2/data/instance:
uuid: "90873f7a558149f094dd37e015a2d8f8"
format_stamp: "Formatted at 2026-08-07 18:16:25 on dist-test-slave-gkw7"
server_key: "11c2a1e3e5528eb5d85d9fe5a4b412a4"
server_key_iv: "9781bbb23881a1c542e42a6b05af0077"
server_key_version: "encryptionkey@0"
I20260807 18:16:25.147085 9388 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/ts-2/wal/instance:
uuid: "90873f7a558149f094dd37e015a2d8f8"
format_stamp: "Formatted at 2026-08-07 18:16:25 on dist-test-slave-gkw7"
server_key: "11c2a1e3e5528eb5d85d9fe5a4b412a4"
server_key_iv: "9781bbb23881a1c542e42a6b05af0077"
server_key_version: "encryptionkey@0"
I20260807 18:16:25.153051 9388 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260807 18:16:25.156873 9410 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:16:25.158447 9388 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:16:25.158674 9388 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/ts-2/wal
uuid: "90873f7a558149f094dd37e015a2d8f8"
format_stamp: "Formatted at 2026-08-07 18:16:25 on dist-test-slave-gkw7"
server_key: "11c2a1e3e5528eb5d85d9fe5a4b412a4"
server_key_iv: "9781bbb23881a1c542e42a6b05af0077"
server_key_version: "encryptionkey@0"
I20260807 18:16:25.158905 9388 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:16:25.162788 9193 heartbeater.cc:503] Master 127.18.154.62:46089 was elected leader, sending a full tablet report...
I20260807 18:16:25.180573 9388 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:16:25.186062 9388 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:16:25.186352 9388 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:16:25.187389 9388 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:16:25.188854 9388 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:16:25.188936 9388 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:16:25.189033 9388 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:16:25.189081 9388 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:16:25.225841 9388 rpc_server.cc:307] RPC server started. Bound to: 127.18.154.3:33869
I20260807 18:16:25.225878 9565 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.154.3:33869 every 8 connection(s)
I20260807 18:16:25.227257 9388 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/ts-2/data/info.pb
I20260807 18:16:25.228626 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 9388
I20260807 18:16:25.228823 19048 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/ts-2/wal/instance
I20260807 18:16:25.229408 19048 external_mini_cluster.cc:1428] Setting key 3be88bc9cf78a49ff277b5cf8e9e388e
Aug 07 18:16:25 dist-test-slave-gkw7 krb5kdc[8868](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126585, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.3@KRBTEST.COM for kudu/127.18.154.62@KRBTEST.COM
I20260807 18:16:25.248661 9571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:25.230280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.3:35637 (local address 127.18.154.62:46089)
0807 18:16:25.230815 (+ 535us) server_negotiation.cc:206] Beginning negotiation
0807 18:16:25.230823 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:16:25.232066 (+ 1243us) server_negotiation.cc:407] Connection header received
0807 18:16:25.234258 (+ 2192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:16:25.234263 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:16:25.234357 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:16:25.234472 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:16:25.236006 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:16:25.236817 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:16:25.238447 (+ 1630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:16:25.238843 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:16:25.238974 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:16:25.242470 (+ 3496us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:16:25.242474 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:16:25.242478 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:16:25.242510 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:16:25.245498 (+ 2988us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:16:25.246291 (+ 793us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:16:25.246297 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:16:25.246301 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:16:25.246428 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:16:25.246875 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:16:25.246880 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:16:25.246883 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:16:25.247176 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:16:25.247346 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0807 18:16:25.248058 (+ 712us) server_negotiation.cc:299] Negotiation successful
0807 18:16:25.248291 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":385,"thread_start_us":266,"threads_started":1}
I20260807 18:16:25.249652 9573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:25.231039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:46089 (local address 127.18.154.3:35637)
0807 18:16:25.231920 (+ 881us) negotiation.cc:107] Waiting for socket to connect
0807 18:16:25.231959 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 18:16:25.234000 (+ 2041us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:16:25.234720 (+ 720us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:16:25.234728 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:16:25.235234 (+ 506us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:16:25.235824 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:16:25.235842 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:16:25.236984 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:16:25.236988 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:16:25.238296 (+ 1308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:16:25.238308 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:16:25.239083 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:16:25.239091 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:16:25.239318 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:16:25.240002 (+ 684us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:16:25.240037 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:16:25.242252 (+ 2215us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:16:25.245691 (+ 3439us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:16:25.245700 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:16:25.245716 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:16:25.246160 (+ 444us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:16:25.246543 (+ 383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:16:25.246548 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:16:25.246552 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:16:25.246751 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:16:25.247315 (+ 564us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:16:25.247323 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:16:25.247668 (+ 345us) client_negotiation.cc:769] Sending connection context
0807 18:16:25.247952 (+ 284us) client_negotiation.cc:240] Negotiation successful
0807 18:16:25.248473 (+ 521us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":569,"thread_start_us":345,"threads_started":1}
I20260807 18:16:25.251189 9568 heartbeater.cc:347] Connected to a master server at 127.18.154.62:46089
I20260807 18:16:25.251470 9568 heartbeater.cc:464] Registering TS with master...
I20260807 18:16:25.252254 9568 heartbeater.cc:511] Master 127.18.154.62:46089 requested a full tablet report, sending...
I20260807 18:16:25.253831 8930 ts_manager.cc:194] Registered new tserver with Master: 90873f7a558149f094dd37e015a2d8f8 (127.18.154.3:33869)
I20260807 18:16:25.254596 8930 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.154.3@KRBTEST.COM'} at 127.18.154.3:35637
I20260807 18:16:25.258297 19048 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 18:16:25.280692 9571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:25.263149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59178 (local address 127.18.154.62:46089)
0807 18:16:25.263327 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0807 18:16:25.263332 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:16:25.263782 (+ 450us) server_negotiation.cc:407] Connection header received
0807 18:16:25.263925 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:16:25.263930 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:16:25.264000 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:16:25.264093 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:16:25.265664 (+ 1571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:16:25.266457 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:16:25.267798 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:16:25.268263 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:16:25.268366 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:16:25.270987 (+ 2621us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:16:25.270992 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:16:25.270996 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:16:25.271031 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:16:25.275069 (+ 4038us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:16:25.276235 (+ 1166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:16:25.276241 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:16:25.276245 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:16:25.276516 (+ 271us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:16:25.277185 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:16:25.277190 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:16:25.277193 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:16:25.277861 (+ 668us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:16:25.279251 (+ 1390us) server_negotiation.cc:1035] Waiting for connection context
0807 18:16:25.279609 (+ 358us) server_negotiation.cc:299] Negotiation successful
0807 18:16:25.280226 (+ 617us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 18:16:25.288318 8930 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59178:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 18:16:25.295683 8930 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:16:25.335530 9586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:25.308679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.3:33869 (local address 127.0.0.1:42722)
0807 18:16:25.321982 (+ 13303us) negotiation.cc:107] Waiting for socket to connect
0807 18:16:25.322063 (+ 81us) client_negotiation.cc:174] Beginning negotiation
0807 18:16:25.322332 (+ 269us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:16:25.325436 (+ 3104us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:16:25.325446 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:16:25.325989 (+ 543us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:16:25.326458 (+ 469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:16:25.326473 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:16:25.329751 (+ 3278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:16:25.329757 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:16:25.332361 (+ 2604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:16:25.332375 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:16:25.334458 (+ 2083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:16:25.334464 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:16:25.334682 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:16:25.334741 (+ 59us) client_negotiation.cc:769] Sending connection context
0807 18:16:25.334876 (+ 135us) client_negotiation.cc:240] Negotiation successful
0807 18:16:25.335088 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12871,"mutex_wait_us":51,"spinlock_wait_cycles":10496,"thread_start_us":13903,"threads_started":1}
I20260807 18:16:25.336195 9592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:25.323994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41886 (local address 127.18.154.2:42929)
0807 18:16:25.327205 (+ 3211us) server_negotiation.cc:206] Beginning negotiation
0807 18:16:25.327211 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:16:25.327235 (+ 24us) server_negotiation.cc:407] Connection header received
0807 18:16:25.327308 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:16:25.327313 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:16:25.327524 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:16:25.327770 (+ 246us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:16:25.328573 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:16:25.329786 (+ 1213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:16:25.333036 (+ 3250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:16:25.334924 (+ 1888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:16:25.335055 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:16:25.335250 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0807 18:16:25.335674 (+ 424us) server_negotiation.cc:299] Negotiation successful
0807 18:16:25.335880 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3033,"thread_start_us":2876,"threads_started":1}
I20260807 18:16:25.336315 9590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:25.323647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.2:42929 (local address 127.0.0.1:41886)
0807 18:16:25.326387 (+ 2740us) negotiation.cc:107] Waiting for socket to connect
0807 18:16:25.326420 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0807 18:16:25.326589 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:16:25.328031 (+ 1442us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:16:25.328035 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:16:25.328056 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:16:25.328366 (+ 310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:16:25.328374 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:16:25.330002 (+ 1628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:16:25.330008 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:16:25.332865 (+ 2857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:16:25.332877 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:16:25.335246 (+ 2369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:16:25.335254 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:16:25.335440 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:16:25.335475 (+ 35us) client_negotiation.cc:769] Sending connection context
0807 18:16:25.335599 (+ 124us) client_negotiation.cc:240] Negotiation successful
0807 18:16:25.335868 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2540,"mutex_wait_us":45,"thread_start_us":2964,"threads_started":1}
I20260807 18:16:25.336711 9589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:25.309193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42722 (local address 127.18.154.3:33869)
0807 18:16:25.324563 (+ 15370us) server_negotiation.cc:206] Beginning negotiation
0807 18:16:25.324577 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 18:16:25.324623 (+ 46us) server_negotiation.cc:407] Connection header received
0807 18:16:25.324759 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:16:25.324771 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:16:25.325157 (+ 386us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:16:25.325430 (+ 273us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:16:25.328036 (+ 2606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:16:25.329604 (+ 1568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:16:25.332524 (+ 2920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:16:25.333907 (+ 1383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:16:25.334041 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:16:25.334307 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0807 18:16:25.336000 (+ 1693us) server_negotiation.cc:299] Negotiation successful
0807 18:16:25.336178 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15128,"thread_start_us":14894,"threads_started":1}
I20260807 18:16:25.337309 9587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:25.308803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.1:34909 (local address 127.0.0.1:34060)
0807 18:16:25.321990 (+ 13187us) negotiation.cc:107] Waiting for socket to connect
0807 18:16:25.322059 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0807 18:16:25.322332 (+ 273us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:16:25.327166 (+ 4834us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:16:25.327171 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:16:25.327194 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:16:25.327606 (+ 412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:16:25.327616 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:16:25.329624 (+ 2008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:16:25.329630 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:16:25.332860 (+ 3230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:16:25.332877 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:16:25.336582 (+ 3705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:16:25.336588 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:16:25.336803 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:16:25.336838 (+ 35us) client_negotiation.cc:769] Sending connection context
0807 18:16:25.336930 (+ 92us) client_negotiation.cc:240] Negotiation successful
0807 18:16:25.337050 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12953,"mutex_wait_us":144,"spinlock_wait_cycles":13952,"thread_start_us":13788,"threads_started":1}
I20260807 18:16:25.338084 9588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:25.309204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34060 (local address 127.18.154.1:34909)
0807 18:16:25.326121 (+ 16917us) server_negotiation.cc:206] Beginning negotiation
0807 18:16:25.326138 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 18:16:25.326169 (+ 31us) server_negotiation.cc:407] Connection header received
0807 18:16:25.326274 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:16:25.326282 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:16:25.326529 (+ 247us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:16:25.326723 (+ 194us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:16:25.327770 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:16:25.329437 (+ 1667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:16:25.333896 (+ 4459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:16:25.335485 (+ 1589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:16:25.335647 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:16:25.336085 (+ 438us) server_negotiation.cc:1035] Waiting for connection context
0807 18:16:25.337531 (+ 1446us) server_negotiation.cc:299] Negotiation successful
0807 18:16:25.337750 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16720,"thread_start_us":17691,"threads_started":1}
I20260807 18:16:25.340466 9286 tablet_service.cc:1467] Processing CreateTablet for tablet 3eace2ff68a64757b86438181d69526e (DEFAULT_TABLE table=test-table [id=06977de8e92a4a5db5ad0d5eecfd1e29]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:16:25.340859 9472 tablet_service.cc:1467] Processing CreateTablet for tablet 3eace2ff68a64757b86438181d69526e (DEFAULT_TABLE table=test-table [id=06977de8e92a4a5db5ad0d5eecfd1e29]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:16:25.341858 9286 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3eace2ff68a64757b86438181d69526e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:16:25.342290 9472 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3eace2ff68a64757b86438181d69526e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:16:25.348728 9100 tablet_service.cc:1467] Processing CreateTablet for tablet 3eace2ff68a64757b86438181d69526e (DEFAULT_TABLE table=test-table [id=06977de8e92a4a5db5ad0d5eecfd1e29]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:16:25.350972 9100 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3eace2ff68a64757b86438181d69526e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:16:25.354594 9593 tablet_bootstrap.cc:492] T 3eace2ff68a64757b86438181d69526e P 3487895bdebd4925b1e0ee7b961c18d9: Bootstrap starting.
I20260807 18:16:25.357921 9595 tablet_bootstrap.cc:492] T 3eace2ff68a64757b86438181d69526e P 90873f7a558149f094dd37e015a2d8f8: Bootstrap starting.
I20260807 18:16:25.358245 9593 tablet_bootstrap.cc:654] T 3eace2ff68a64757b86438181d69526e P 3487895bdebd4925b1e0ee7b961c18d9: Neither blocks nor log segments found. Creating new log.
I20260807 18:16:25.359719 9593 log.cc:824] T 3eace2ff68a64757b86438181d69526e P 3487895bdebd4925b1e0ee7b961c18d9: Log is configured to *not* fsync() on all Append() calls
I20260807 18:16:25.361716 9595 tablet_bootstrap.cc:654] T 3eace2ff68a64757b86438181d69526e P 90873f7a558149f094dd37e015a2d8f8: Neither blocks nor log segments found. Creating new log.
I20260807 18:16:25.363049 9595 log.cc:824] T 3eace2ff68a64757b86438181d69526e P 90873f7a558149f094dd37e015a2d8f8: Log is configured to *not* fsync() on all Append() calls
I20260807 18:16:25.363379 9593 tablet_bootstrap.cc:492] T 3eace2ff68a64757b86438181d69526e P 3487895bdebd4925b1e0ee7b961c18d9: No bootstrap required, opened a new log
I20260807 18:16:25.363741 9593 ts_tablet_manager.cc:1397] T 3eace2ff68a64757b86438181d69526e P 3487895bdebd4925b1e0ee7b961c18d9: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260807 18:16:25.366619 9595 tablet_bootstrap.cc:492] T 3eace2ff68a64757b86438181d69526e P 90873f7a558149f094dd37e015a2d8f8: No bootstrap required, opened a new log
I20260807 18:16:25.366894 9595 ts_tablet_manager.cc:1397] T 3eace2ff68a64757b86438181d69526e P 90873f7a558149f094dd37e015a2d8f8: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.004s
I20260807 18:16:25.367962 9600 tablet_bootstrap.cc:492] T 3eace2ff68a64757b86438181d69526e P b3f50a9f91964b79b66d73c38ac9ab47: Bootstrap starting.
I20260807 18:16:25.371409 9600 tablet_bootstrap.cc:654] T 3eace2ff68a64757b86438181d69526e P b3f50a9f91964b79b66d73c38ac9ab47: Neither blocks nor log segments found. Creating new log.
I20260807 18:16:25.372637 9600 log.cc:824] T 3eace2ff68a64757b86438181d69526e P b3f50a9f91964b79b66d73c38ac9ab47: Log is configured to *not* fsync() on all Append() calls
I20260807 18:16:25.374945 9593 raft_consensus.cc:348] T 3eace2ff68a64757b86438181d69526e P 3487895bdebd4925b1e0ee7b961c18d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90873f7a558149f094dd37e015a2d8f8" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 33869 } } peers { permanent_uuid: "b3f50a9f91964b79b66d73c38ac9ab47" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 34909 } } peers { permanent_uuid: "3487895bdebd4925b1e0ee7b961c18d9" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 42929 } }
I20260807 18:16:25.375398 9593 raft_consensus.cc:374] T 3eace2ff68a64757b86438181d69526e P 3487895bdebd4925b1e0ee7b961c18d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:16:25.375499 9593 raft_consensus.cc:729] T 3eace2ff68a64757b86438181d69526e P 3487895bdebd4925b1e0ee7b961c18d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3487895bdebd4925b1e0ee7b961c18d9, State: Initialized, Role: FOLLOWER
I20260807 18:16:25.375602 9600 tablet_bootstrap.cc:492] T 3eace2ff68a64757b86438181d69526e P b3f50a9f91964b79b66d73c38ac9ab47: No bootstrap required, opened a new log
I20260807 18:16:25.375952 9600 ts_tablet_manager.cc:1397] T 3eace2ff68a64757b86438181d69526e P b3f50a9f91964b79b66d73c38ac9ab47: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260807 18:16:25.376176 9593 consensus_queue.cc:260] T 3eace2ff68a64757b86438181d69526e P 3487895bdebd4925b1e0ee7b961c18d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90873f7a558149f094dd37e015a2d8f8" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 33869 } } peers { permanent_uuid: "b3f50a9f91964b79b66d73c38ac9ab47" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 34909 } } peers { permanent_uuid: "3487895bdebd4925b1e0ee7b961c18d9" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 42929 } }
I20260807 18:16:25.377749 9595 raft_consensus.cc:348] T 3eace2ff68a64757b86438181d69526e P 90873f7a558149f094dd37e015a2d8f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90873f7a558149f094dd37e015a2d8f8" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 33869 } } peers { permanent_uuid: "b3f50a9f91964b79b66d73c38ac9ab47" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 34909 } } peers { permanent_uuid: "3487895bdebd4925b1e0ee7b961c18d9" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 42929 } }
I20260807 18:16:25.378202 9593 ts_tablet_manager.cc:1428] T 3eace2ff68a64757b86438181d69526e P 3487895bdebd4925b1e0ee7b961c18d9: Time spent starting tablet: real 0.014s user 0.011s sys 0.004s
I20260807 18:16:25.378216 9595 raft_consensus.cc:374] T 3eace2ff68a64757b86438181d69526e P 90873f7a558149f094dd37e015a2d8f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:16:25.378311 9595 raft_consensus.cc:729] T 3eace2ff68a64757b86438181d69526e P 90873f7a558149f094dd37e015a2d8f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90873f7a558149f094dd37e015a2d8f8, State: Initialized, Role: FOLLOWER
I20260807 18:16:25.378453 9376 heartbeater.cc:503] Master 127.18.154.62:46089 was elected leader, sending a full tablet report...
I20260807 18:16:25.378916 9595 consensus_queue.cc:260] T 3eace2ff68a64757b86438181d69526e P 90873f7a558149f094dd37e015a2d8f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90873f7a558149f094dd37e015a2d8f8" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 33869 } } peers { permanent_uuid: "b3f50a9f91964b79b66d73c38ac9ab47" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 34909 } } peers { permanent_uuid: "3487895bdebd4925b1e0ee7b961c18d9" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 42929 } }
I20260807 18:16:25.380162 9568 heartbeater.cc:503] Master 127.18.154.62:46089 was elected leader, sending a full tablet report...
I20260807 18:16:25.380857 9595 ts_tablet_manager.cc:1428] T 3eace2ff68a64757b86438181d69526e P 90873f7a558149f094dd37e015a2d8f8: Time spent starting tablet: real 0.014s user 0.007s sys 0.007s
I20260807 18:16:25.387037 9600 raft_consensus.cc:348] T 3eace2ff68a64757b86438181d69526e P b3f50a9f91964b79b66d73c38ac9ab47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90873f7a558149f094dd37e015a2d8f8" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 33869 } } peers { permanent_uuid: "b3f50a9f91964b79b66d73c38ac9ab47" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 34909 } } peers { permanent_uuid: "3487895bdebd4925b1e0ee7b961c18d9" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 42929 } }
I20260807 18:16:25.387389 9600 raft_consensus.cc:374] T 3eace2ff68a64757b86438181d69526e P b3f50a9f91964b79b66d73c38ac9ab47 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:16:25.387466 9600 raft_consensus.cc:729] T 3eace2ff68a64757b86438181d69526e P b3f50a9f91964b79b66d73c38ac9ab47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3f50a9f91964b79b66d73c38ac9ab47, State: Initialized, Role: FOLLOWER
I20260807 18:16:25.387959 9600 consensus_queue.cc:260] T 3eace2ff68a64757b86438181d69526e P b3f50a9f91964b79b66d73c38ac9ab47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90873f7a558149f094dd37e015a2d8f8" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 33869 } } peers { permanent_uuid: "b3f50a9f91964b79b66d73c38ac9ab47" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 34909 } } peers { permanent_uuid: "3487895bdebd4925b1e0ee7b961c18d9" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 42929 } }
I20260807 18:16:25.389446 9600 ts_tablet_manager.cc:1428] T 3eace2ff68a64757b86438181d69526e P b3f50a9f91964b79b66d73c38ac9ab47: Time spent starting tablet: real 0.013s user 0.004s sys 0.010s
I20260807 18:16:25.419770 9602 raft_consensus.cc:482] T 3eace2ff68a64757b86438181d69526e P 3487895bdebd4925b1e0ee7b961c18d9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:16:25.420121 9602 raft_consensus.cc:504] T 3eace2ff68a64757b86438181d69526e P 3487895bdebd4925b1e0ee7b961c18d9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90873f7a558149f094dd37e015a2d8f8" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 33869 } } peers { permanent_uuid: "b3f50a9f91964b79b66d73c38ac9ab47" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 34909 } } peers { permanent_uuid: "3487895bdebd4925b1e0ee7b961c18d9" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 42929 } }
I20260807 18:16:25.422080 9602 leader_election.cc:288] T 3eace2ff68a64757b86438181d69526e P 3487895bdebd4925b1e0ee7b961c18d9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 90873f7a558149f094dd37e015a2d8f8 (127.18.154.3:33869), b3f50a9f91964b79b66d73c38ac9ab47 (127.18.154.1:34909)
W20260807 18:16:25.427851 9380 tablet.cc:2369] T 3eace2ff68a64757b86438181d69526e P 3487895bdebd4925b1e0ee7b961c18d9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:16:25.430272 9588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:25.422806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.2:41147 (local address 127.18.154.1:34909)
0807 18:16:25.423014 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0807 18:16:25.423020 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:16:25.423620 (+ 600us) server_negotiation.cc:407] Connection header received
0807 18:16:25.423838 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:16:25.423842 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:16:25.423901 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:16:25.423999 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:16:25.424645 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:16:25.425366 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:16:25.427829 (+ 2463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:16:25.429095 (+ 1266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:16:25.429245 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:16:25.429319 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0807 18:16:25.429901 (+ 582us) server_negotiation.cc:299] Negotiation successful
0807 18:16:25.430025 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260807 18:16:25.430549 9607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:25.422629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.1:34909 (local address 127.18.154.2:41147)
0807 18:16:25.423489 (+ 860us) negotiation.cc:107] Waiting for socket to connect
0807 18:16:25.423509 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 18:16:25.423694 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:16:25.424019 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:16:25.424023 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:16:25.424066 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:16:25.424481 (+ 415us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:16:25.424490 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:16:25.425511 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:16:25.425516 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:16:25.427568 (+ 2052us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:16:25.427585 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:16:25.429319 (+ 1734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:16:25.429325 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:16:25.429631 (+ 306us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:16:25.429763 (+ 132us) client_negotiation.cc:769] Sending connection context
0807 18:16:25.429865 (+ 102us) client_negotiation.cc:240] Negotiation successful
0807 18:16:25.430075 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":725,"mutex_wait_us":211,"thread_start_us":531,"threads_started":1}
I20260807 18:16:25.430605 9606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:25.422290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.3:33869 (local address 127.18.154.2:50895)
0807 18:16:25.423351 (+ 1061us) negotiation.cc:107] Waiting for socket to connect
0807 18:16:25.423377 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0807 18:16:25.423539 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:16:25.423915 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:16:25.423919 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:16:25.423944 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:16:25.424355 (+ 411us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:16:25.424368 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:16:25.425432 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:16:25.425438 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:16:25.427543 (+ 2105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:16:25.427556 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:16:25.429319 (+ 1763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:16:25.429325 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:16:25.429644 (+ 319us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:16:25.429763 (+ 119us) client_negotiation.cc:769] Sending connection context
0807 18:16:25.429865 (+ 102us) client_negotiation.cc:240] Negotiation successful
0807 18:16:25.430075 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":877,"mutex_wait_us":78,"thread_start_us":599,"threads_started":1}
I20260807 18:16:25.430996 9589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:25.422452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.2:50895 (local address 127.18.154.3:33869)
0807 18:16:25.422671 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0807 18:16:25.422680 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:16:25.423505 (+ 825us) server_negotiation.cc:407] Connection header received
0807 18:16:25.423719 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:16:25.423725 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:16:25.423790 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:16:25.423897 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:16:25.424530 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:16:25.425289 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:16:25.427705 (+ 2416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:16:25.429005 (+ 1300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:16:25.429204 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:16:25.429285 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0807 18:16:25.430539 (+ 1254us) server_negotiation.cc:299] Negotiation successful
0807 18:16:25.430737 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260807 18:16:25.431902 9503 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3eace2ff68a64757b86438181d69526e" candidate_uuid: "3487895bdebd4925b1e0ee7b961c18d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90873f7a558149f094dd37e015a2d8f8" is_pre_election: true
I20260807 18:16:25.431891 9127 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3eace2ff68a64757b86438181d69526e" candidate_uuid: "3487895bdebd4925b1e0ee7b961c18d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3f50a9f91964b79b66d73c38ac9ab47" is_pre_election: true
I20260807 18:16:25.432417 9127 raft_consensus.cc:2454] T 3eace2ff68a64757b86438181d69526e P b3f50a9f91964b79b66d73c38ac9ab47 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3487895bdebd4925b1e0ee7b961c18d9 in term 0.
I20260807 18:16:25.432417 9503 raft_consensus.cc:2454] T 3eace2ff68a64757b86438181d69526e P 90873f7a558149f094dd37e015a2d8f8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3487895bdebd4925b1e0ee7b961c18d9 in term 0.
I20260807 18:16:25.433084 9223 leader_election.cc:302] T 3eace2ff68a64757b86438181d69526e P 3487895bdebd4925b1e0ee7b961c18d9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3487895bdebd4925b1e0ee7b961c18d9, b3f50a9f91964b79b66d73c38ac9ab47; no voters:
I20260807 18:16:25.433506 9602 raft_consensus.cc:2781] T 3eace2ff68a64757b86438181d69526e P 3487895bdebd4925b1e0ee7b961c18d9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:16:25.433681 9602 raft_consensus.cc:482] T 3eace2ff68a64757b86438181d69526e P 3487895bdebd4925b1e0ee7b961c18d9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:16:25.433794 9602 raft_consensus.cc:3037] T 3eace2ff68a64757b86438181d69526e P 3487895bdebd4925b1e0ee7b961c18d9 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:16:25.436022 9602 raft_consensus.cc:504] T 3eace2ff68a64757b86438181d69526e P 3487895bdebd4925b1e0ee7b961c18d9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90873f7a558149f094dd37e015a2d8f8" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 33869 } } peers { permanent_uuid: "b3f50a9f91964b79b66d73c38ac9ab47" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 34909 } } peers { permanent_uuid: "3487895bdebd4925b1e0ee7b961c18d9" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 42929 } }
I20260807 18:16:25.436682 9602 leader_election.cc:288] T 3eace2ff68a64757b86438181d69526e P 3487895bdebd4925b1e0ee7b961c18d9 [CANDIDATE]: Term 1 election: Requested vote from peers 90873f7a558149f094dd37e015a2d8f8 (127.18.154.3:33869), b3f50a9f91964b79b66d73c38ac9ab47 (127.18.154.1:34909)
I20260807 18:16:25.437181 9503 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3eace2ff68a64757b86438181d69526e" candidate_uuid: "3487895bdebd4925b1e0ee7b961c18d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90873f7a558149f094dd37e015a2d8f8"
I20260807 18:16:25.437238 9127 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3eace2ff68a64757b86438181d69526e" candidate_uuid: "3487895bdebd4925b1e0ee7b961c18d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3f50a9f91964b79b66d73c38ac9ab47"
I20260807 18:16:25.437427 9127 raft_consensus.cc:3037] T 3eace2ff68a64757b86438181d69526e P b3f50a9f91964b79b66d73c38ac9ab47 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:16:25.437418 9503 raft_consensus.cc:3037] T 3eace2ff68a64757b86438181d69526e P 90873f7a558149f094dd37e015a2d8f8 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:16:25.440773 9127 raft_consensus.cc:2454] T 3eace2ff68a64757b86438181d69526e P b3f50a9f91964b79b66d73c38ac9ab47 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3487895bdebd4925b1e0ee7b961c18d9 in term 1.
I20260807 18:16:25.440773 9503 raft_consensus.cc:2454] T 3eace2ff68a64757b86438181d69526e P 90873f7a558149f094dd37e015a2d8f8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3487895bdebd4925b1e0ee7b961c18d9 in term 1.
I20260807 18:16:25.441274 9223 leader_election.cc:302] T 3eace2ff68a64757b86438181d69526e P 3487895bdebd4925b1e0ee7b961c18d9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3487895bdebd4925b1e0ee7b961c18d9, b3f50a9f91964b79b66d73c38ac9ab47; no voters:
I20260807 18:16:25.441603 9602 raft_consensus.cc:2781] T 3eace2ff68a64757b86438181d69526e P 3487895bdebd4925b1e0ee7b961c18d9 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:16:25.442059 9602 raft_consensus.cc:686] T 3eace2ff68a64757b86438181d69526e P 3487895bdebd4925b1e0ee7b961c18d9 [term 1 LEADER]: Becoming Leader. State: Replica: 3487895bdebd4925b1e0ee7b961c18d9, State: Running, Role: LEADER
I20260807 18:16:25.442726 9602 consensus_queue.cc:237] T 3eace2ff68a64757b86438181d69526e P 3487895bdebd4925b1e0ee7b961c18d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90873f7a558149f094dd37e015a2d8f8" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 33869 } } peers { permanent_uuid: "b3f50a9f91964b79b66d73c38ac9ab47" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 34909 } } peers { permanent_uuid: "3487895bdebd4925b1e0ee7b961c18d9" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 42929 } }
I20260807 18:16:25.448376 8930 catalog_manager.cc:5495] T 3eace2ff68a64757b86438181d69526e P 3487895bdebd4925b1e0ee7b961c18d9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3487895bdebd4925b1e0ee7b961c18d9 (127.18.154.2). New cstate: current_term: 1 leader_uuid: "3487895bdebd4925b1e0ee7b961c18d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90873f7a558149f094dd37e015a2d8f8" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 33869 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b3f50a9f91964b79b66d73c38ac9ab47" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3487895bdebd4925b1e0ee7b961c18d9" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 42929 } health_report { overall_health: HEALTHY } } }
W20260807 18:16:25.479202 9570 tablet.cc:2369] T 3eace2ff68a64757b86438181d69526e P 90873f7a558149f094dd37e015a2d8f8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:16:25.493777 9592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:25.488435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41892 (local address 127.18.154.2:42929)
0807 18:16:25.488625 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0807 18:16:25.488629 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:16:25.488647 (+ 18us) server_negotiation.cc:407] Connection header received
0807 18:16:25.488734 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:16:25.488738 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:16:25.488825 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:16:25.488956 (+ 131us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:16:25.489339 (+ 383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:16:25.490174 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:16:25.491730 (+ 1556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:16:25.492123 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:16:25.492222 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:16:25.492677 (+ 455us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:16:25.493212 (+ 535us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:16:25.493324 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0807 18:16:25.493430 (+ 106us) server_negotiation.cc:299] Negotiation successful
0807 18:16:25.493506 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 18:16:25.502305 9503 raft_consensus.cc:1261] T 3eace2ff68a64757b86438181d69526e P 90873f7a558149f094dd37e015a2d8f8 [term 1 FOLLOWER]: Refusing update from remote peer 3487895bdebd4925b1e0ee7b961c18d9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:16:25.502389 9127 raft_consensus.cc:1261] T 3eace2ff68a64757b86438181d69526e P b3f50a9f91964b79b66d73c38ac9ab47 [term 1 FOLLOWER]: Refusing update from remote peer 3487895bdebd4925b1e0ee7b961c18d9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:16:25.503525 9608 consensus_queue.cc:1035] T 3eace2ff68a64757b86438181d69526e P 3487895bdebd4925b1e0ee7b961c18d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3f50a9f91964b79b66d73c38ac9ab47" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:16:25.503893 9602 consensus_queue.cc:1035] T 3eace2ff68a64757b86438181d69526e P 3487895bdebd4925b1e0ee7b961c18d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90873f7a558149f094dd37e015a2d8f8" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 33869 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:16:25.530117 8930 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59178:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:16:25.530457 8930 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59178:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:16:25.534468 8930 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 74b7e49ff11046dba46d21f00d9fd15a: Sending DeleteTablet for 3 replicas of tablet 3eace2ff68a64757b86438181d69526e
I20260807 18:16:25.535966 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 9011
I20260807 18:16:25.535890 9472 tablet_service.cc:1514] Processing DeleteTablet for tablet 3eace2ff68a64757b86438181d69526e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:16:25 UTC) from {username='kudu', principal='kudu/127.18.154.62@KRBTEST.COM'} at 127.0.0.1:42722
I20260807 18:16:25.536175 9286 tablet_service.cc:1514] Processing DeleteTablet for tablet 3eace2ff68a64757b86438181d69526e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:16:25 UTC) from {username='kudu', principal='kudu/127.18.154.62@KRBTEST.COM'} at 127.0.0.1:41886
I20260807 18:16:25.536661 9100 tablet_service.cc:1514] Processing DeleteTablet for tablet 3eace2ff68a64757b86438181d69526e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:16:25 UTC) from {username='kudu', principal='kudu/127.18.154.62@KRBTEST.COM'} at 127.0.0.1:34060
I20260807 18:16:25.548465 9625 tablet_replica.cc:330] T 3eace2ff68a64757b86438181d69526e P 90873f7a558149f094dd37e015a2d8f8: stopping tablet replica
I20260807 18:16:25.549032 9625 raft_consensus.cc:2229] T 3eace2ff68a64757b86438181d69526e P 90873f7a558149f094dd37e015a2d8f8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:16:25.550102 9626 tablet_replica.cc:330] T 3eace2ff68a64757b86438181d69526e P 3487895bdebd4925b1e0ee7b961c18d9: stopping tablet replica
I20260807 18:16:25.551187 9627 tablet_replica.cc:330] T 3eace2ff68a64757b86438181d69526e P b3f50a9f91964b79b66d73c38ac9ab47: stopping tablet replica
I20260807 18:16:25.551218 9625 raft_consensus.cc:2258] T 3eace2ff68a64757b86438181d69526e P 90873f7a558149f094dd37e015a2d8f8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:16:25.550590 9626 raft_consensus.cc:2229] T 3eace2ff68a64757b86438181d69526e P 3487895bdebd4925b1e0ee7b961c18d9 [term 1 LEADER]: Raft consensus shutting down.
I20260807 18:16:25.551872 9627 raft_consensus.cc:2229] T 3eace2ff68a64757b86438181d69526e P b3f50a9f91964b79b66d73c38ac9ab47 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:16:25.552433 9626 raft_consensus.cc:2258] T 3eace2ff68a64757b86438181d69526e P 3487895bdebd4925b1e0ee7b961c18d9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:16:25.552589 9627 raft_consensus.cc:2258] T 3eace2ff68a64757b86438181d69526e P b3f50a9f91964b79b66d73c38ac9ab47 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:16:25.554282 9625 ts_tablet_manager.cc:1905] T 3eace2ff68a64757b86438181d69526e P 90873f7a558149f094dd37e015a2d8f8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:16:25.554641 9626 ts_tablet_manager.cc:1905] T 3eace2ff68a64757b86438181d69526e P 3487895bdebd4925b1e0ee7b961c18d9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:16:25.561909 9625 ts_tablet_manager.cc:1918] T 3eace2ff68a64757b86438181d69526e P 90873f7a558149f094dd37e015a2d8f8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
Aug 07 18:16:25 dist-test-slave-gkw7 krb5kdc[8868](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126582, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.154.1@KRBTEST.COM
I20260807 18:16:25.562096 9625 log.cc:1196] T 3eace2ff68a64757b86438181d69526e P 90873f7a558149f094dd37e015a2d8f8: Deleting WAL directory at /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/ts-2/wal/wals/3eace2ff68a64757b86438181d69526e
I20260807 18:16:25.562644 9625 ts_tablet_manager.cc:1939] T 3eace2ff68a64757b86438181d69526e P 90873f7a558149f094dd37e015a2d8f8: Deleting consensus metadata
I20260807 18:16:25.563037 9627 ts_tablet_manager.cc:1905] T 3eace2ff68a64757b86438181d69526e P b3f50a9f91964b79b66d73c38ac9ab47: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:16:25.564994 8912 catalog_manager.cc:4841] TS 90873f7a558149f094dd37e015a2d8f8 (127.18.154.3:33869): tablet 3eace2ff68a64757b86438181d69526e (table test-table [id=06977de8e92a4a5db5ad0d5eecfd1e29]) successfully deleted
I20260807 18:16:25.567407 9626 ts_tablet_manager.cc:1918] T 3eace2ff68a64757b86438181d69526e P 3487895bdebd4925b1e0ee7b961c18d9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:16:25.567598 9626 log.cc:1196] T 3eace2ff68a64757b86438181d69526e P 3487895bdebd4925b1e0ee7b961c18d9: Deleting WAL directory at /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/ts-1/wal/wals/3eace2ff68a64757b86438181d69526e
I20260807 18:16:25.568195 9626 ts_tablet_manager.cc:1939] T 3eace2ff68a64757b86438181d69526e P 3487895bdebd4925b1e0ee7b961c18d9: Deleting consensus metadata
I20260807 18:16:25.570098 8912 catalog_manager.cc:4841] TS 3487895bdebd4925b1e0ee7b961c18d9 (127.18.154.2:42929): tablet 3eace2ff68a64757b86438181d69526e (table test-table [id=06977de8e92a4a5db5ad0d5eecfd1e29]) successfully deleted
I20260807 18:16:25.571050 9627 ts_tablet_manager.cc:1918] T 3eace2ff68a64757b86438181d69526e P b3f50a9f91964b79b66d73c38ac9ab47: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:16:25.571251 9627 log.cc:1196] T 3eace2ff68a64757b86438181d69526e P b3f50a9f91964b79b66d73c38ac9ab47: Deleting WAL directory at /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/ts-0/wal/wals/3eace2ff68a64757b86438181d69526e
I20260807 18:16:25.571789 9627 ts_tablet_manager.cc:1939] T 3eace2ff68a64757b86438181d69526e P b3f50a9f91964b79b66d73c38ac9ab47: Deleting consensus metadata
I20260807 18:16:25.573238 9588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:25.537440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34076 (local address 127.18.154.1:34909)
0807 18:16:25.537677 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0807 18:16:25.537687 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:16:25.548495 (+ 10808us) server_negotiation.cc:407] Connection header received
0807 18:16:25.549087 (+ 592us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:16:25.549093 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:16:25.549207 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:16:25.549294 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:16:25.552198 (+ 2904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:16:25.553413 (+ 1215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:16:25.555770 (+ 2357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:16:25.556451 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:16:25.556552 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:16:25.563680 (+ 7128us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:16:25.563695 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:16:25.563713 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:16:25.563809 (+ 96us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:16:25.568754 (+ 4945us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:16:25.570326 (+ 1572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:16:25.570337 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:16:25.570341 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:16:25.570517 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:16:25.571086 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:16:25.571093 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:16:25.571097 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:16:25.571602 (+ 505us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:16:25.571941 (+ 339us) server_negotiation.cc:1035] Waiting for connection context
0807 18:16:25.572541 (+ 600us) server_negotiation.cc:299] Negotiation successful
0807 18:16:25.572801 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260807 18:16:25.573704 8913 catalog_manager.cc:4841] TS b3f50a9f91964b79b66d73c38ac9ab47 (127.18.154.1:34909): tablet 3eace2ff68a64757b86438181d69526e (table test-table [id=06977de8e92a4a5db5ad0d5eecfd1e29]) successfully deleted
I20260807 18:16:25.574193 9187 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:16:25.666607 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 9011
I20260807 18:16:25.702242 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 9199
Aug 07 18:16:25 dist-test-slave-gkw7 krb5kdc[8868](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126582, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.154.2@KRBTEST.COM
I20260807 18:16:25.727862 9592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:25.703130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41898 (local address 127.18.154.2:42929)
0807 18:16:25.703320 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0807 18:16:25.703324 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:16:25.703358 (+ 34us) server_negotiation.cc:407] Connection header received
0807 18:16:25.703578 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:16:25.703582 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:16:25.703658 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:16:25.703746 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:16:25.705792 (+ 2046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:16:25.706730 (+ 938us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:16:25.709087 (+ 2357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:16:25.709658 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:16:25.709821 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:16:25.720208 (+ 10387us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:16:25.720241 (+ 33us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:16:25.720265 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:16:25.720329 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:16:25.724220 (+ 3891us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:16:25.725324 (+ 1104us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:16:25.725333 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:16:25.725336 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:16:25.725501 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:16:25.726043 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:16:25.726047 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:16:25.726050 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:16:25.726576 (+ 526us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:16:25.726912 (+ 336us) server_negotiation.cc:1035] Waiting for connection context
0807 18:16:25.727299 (+ 387us) server_negotiation.cc:299] Negotiation successful
0807 18:16:25.727486 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260807 18:16:25.728663 9368 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:16:25.846035 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 9199
I20260807 18:16:25.873201 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 9388
Aug 07 18:16:25 dist-test-slave-gkw7 krb5kdc[8868](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126582, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.154.3@KRBTEST.COM
I20260807 18:16:25.890070 9589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:25.874135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42738 (local address 127.18.154.3:33869)
0807 18:16:25.874316 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0807 18:16:25.874322 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:16:25.874461 (+ 139us) server_negotiation.cc:407] Connection header received
0807 18:16:25.874625 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:16:25.874640 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:16:25.874709 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:16:25.874798 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:16:25.876385 (+ 1587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:16:25.877173 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:16:25.878770 (+ 1597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:16:25.879189 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:16:25.879292 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:16:25.884021 (+ 4729us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:16:25.884039 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:16:25.884065 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:16:25.884110 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:16:25.886980 (+ 2870us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:16:25.887764 (+ 784us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:16:25.887773 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:16:25.887775 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:16:25.887924 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:16:25.888331 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:16:25.888337 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:16:25.888341 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:16:25.888723 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:16:25.889038 (+ 315us) server_negotiation.cc:1035] Waiting for connection context
0807 18:16:25.889391 (+ 353us) server_negotiation.cc:299] Negotiation successful
0807 18:16:25.889670 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260807 18:16:25.890739 9559 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:16:26.007419 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 9388
I20260807 18:16:26.034989 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 8885
I20260807 18:16:26.036141 8972 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:16:26.149349 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 8885
I20260807 18:16:46.181517 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.154.62:46089
--webserver_interface=127.18.154.62
--webserver_port=44395
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.62.keytab
--principal=kudu/127.18.154.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.154.20:44143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.154.62:46089
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:16:46.558646 9637 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:16:46.558947 9637 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:16:46.559039 9637 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:16:46.566614 9637 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260807 18:16:46.566725 9637 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260807 18:16:46.566764 9637 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260807 18:16:46.566811 9637 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:16:46.566862 9637 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:16:46.566931 9637 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:16:46.566977 9637 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:16:46.567024 9637 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:16:46.577256 9637 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.154.20:44143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.18.154.62:46089
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.62.keytab
--principal=kudu/127.18.154.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.154.62:46089
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.154.62
--webserver_port=44395
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:16:46.578681 9637 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:16:46.580024 9637 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:16:46.607358 9648 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:16:46.607460 9644 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:16:46.607957 9651 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:16:46.608615 9637 server_base.cc:1034] running on GCE node
I20260807 18:16:46.609505 9637 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:16:46.611202 9637 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:16:46.612447 9637 hybrid_clock.cc:648] HybridClock initialized: now 1786126606612352 us; error 65 us; skew 500 ppm
I20260807 18:16:46.612840 9637 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:16:46 dist-test-slave-gkw7 krb5kdc[8868](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126606, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:16:46.616686 9637 init.cc:377] Logged in from keytab as kudu/127.18.154.62@KRBTEST.COM (short username kudu)
I20260807 18:16:46.619133 9637 webserver.cc:460] Webserver started at http://127.18.154.62:44395/ using document root <none> and password file <none>
I20260807 18:16:46.619925 9637 fs_manager.cc:359] Metadata directory not provided
I20260807 18:16:46.620038 9637 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:16:46.626454 9637 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260807 18:16:46.629671 9661 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:16:46.631474 9637 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 18:16:46.631687 9637 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/master-0/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/master-0/wal
uuid: "74b7e49ff11046dba46d21f00d9fd15a"
format_stamp: "Formatted at 2026-08-07 18:16:23 on dist-test-slave-gkw7"
server_key: "25521df0306039dcaea0a5175ac320c4"
server_key_iv: "3c597486b4bad2870224c9e0ab5dee35"
server_key_version: "encryptionkey@0"
I20260807 18:16:46.632807 9637 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:16:46.655400 9637 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:16:46.660326 9637 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:16:46.660677 9637 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:16:46.681058 9637 rpc_server.cc:307] RPC server started. Bound to: 127.18.154.62:46089
I20260807 18:16:46.681082 9731 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.154.62:46089 every 8 connection(s)
I20260807 18:16:46.682822 9637 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
I20260807 18:16:46.687706 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 9637
I20260807 18:16:46.689074 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.154.1:34909
--local_ip_for_outbound_sockets=127.18.154.1
--tserver_master_addrs=127.18.154.62:46089
--webserver_port=40111
--webserver_interface=127.18.154.1
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.1.keytab
--principal=kudu/127.18.154.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.154.20:44143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 18:16:46.692551 9734 sys_catalog.cc:263] Verifying existing consensus state
I20260807 18:16:46.698632 9734 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 74b7e49ff11046dba46d21f00d9fd15a: Bootstrap starting.
I20260807 18:16:46.722472 9734 log.cc:824] T 00000000000000000000000000000000 P 74b7e49ff11046dba46d21f00d9fd15a: Log is configured to *not* fsync() on all Append() calls
I20260807 18:16:46.736271 9734 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 74b7e49ff11046dba46d21f00d9fd15a: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 18:16:46.736905 9734 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 74b7e49ff11046dba46d21f00d9fd15a: Bootstrap complete.
I20260807 18:16:46.746750 9734 raft_consensus.cc:348] T 00000000000000000000000000000000 P 74b7e49ff11046dba46d21f00d9fd15a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74b7e49ff11046dba46d21f00d9fd15a" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 46089 } }
I20260807 18:16:46.747601 9734 raft_consensus.cc:729] T 00000000000000000000000000000000 P 74b7e49ff11046dba46d21f00d9fd15a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74b7e49ff11046dba46d21f00d9fd15a, State: Initialized, Role: FOLLOWER
I20260807 18:16:46.748306 9734 consensus_queue.cc:260] T 00000000000000000000000000000000 P 74b7e49ff11046dba46d21f00d9fd15a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74b7e49ff11046dba46d21f00d9fd15a" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 46089 } }
I20260807 18:16:46.748557 9734 raft_consensus.cc:388] T 00000000000000000000000000000000 P 74b7e49ff11046dba46d21f00d9fd15a [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:16:46.748687 9734 raft_consensus.cc:482] T 00000000000000000000000000000000 P 74b7e49ff11046dba46d21f00d9fd15a [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:16:46.748899 9734 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 74b7e49ff11046dba46d21f00d9fd15a [term 1 FOLLOWER]: Advancing to term 2
I20260807 18:16:46.752880 9734 raft_consensus.cc:504] T 00000000000000000000000000000000 P 74b7e49ff11046dba46d21f00d9fd15a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74b7e49ff11046dba46d21f00d9fd15a" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 46089 } }
I20260807 18:16:46.753449 9734 leader_election.cc:302] T 00000000000000000000000000000000 P 74b7e49ff11046dba46d21f00d9fd15a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 74b7e49ff11046dba46d21f00d9fd15a; no voters:
I20260807 18:16:46.754336 9734 leader_election.cc:288] T 00000000000000000000000000000000 P 74b7e49ff11046dba46d21f00d9fd15a [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 18:16:46.754475 9739 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 74b7e49ff11046dba46d21f00d9fd15a [term 2 FOLLOWER]: Leader election won for term 2
I20260807 18:16:46.755767 9739 raft_consensus.cc:686] T 00000000000000000000000000000000 P 74b7e49ff11046dba46d21f00d9fd15a [term 2 LEADER]: Becoming Leader. State: Replica: 74b7e49ff11046dba46d21f00d9fd15a, State: Running, Role: LEADER
I20260807 18:16:46.756341 9739 consensus_queue.cc:237] T 00000000000000000000000000000000 P 74b7e49ff11046dba46d21f00d9fd15a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74b7e49ff11046dba46d21f00d9fd15a" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 46089 } }
I20260807 18:16:46.756884 9734 sys_catalog.cc:564] T 00000000000000000000000000000000 P 74b7e49ff11046dba46d21f00d9fd15a [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:16:46.759140 9741 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74b7e49ff11046dba46d21f00d9fd15a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 74b7e49ff11046dba46d21f00d9fd15a. Latest consensus state: current_term: 2 leader_uuid: "74b7e49ff11046dba46d21f00d9fd15a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74b7e49ff11046dba46d21f00d9fd15a" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 46089 } } }
I20260807 18:16:46.759527 9741 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74b7e49ff11046dba46d21f00d9fd15a [sys.catalog]: This master's current role is: LEADER
I20260807 18:16:46.759192 9740 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74b7e49ff11046dba46d21f00d9fd15a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "74b7e49ff11046dba46d21f00d9fd15a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74b7e49ff11046dba46d21f00d9fd15a" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 46089 } } }
I20260807 18:16:46.759673 9740 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74b7e49ff11046dba46d21f00d9fd15a [sys.catalog]: This master's current role is: LEADER
I20260807 18:16:46.769179 9746 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:16:46.778267 9746 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:16:46.779757 9746 catalog_manager.cc:1259] Loaded cluster ID: 64638edcd2e6457cb96fce940f3801a9
I20260807 18:16:46.779927 9746 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:16:46.784183 9746 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:16:46.785811 9746 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 74b7e49ff11046dba46d21f00d9fd15a: Loaded TSK: 0
I20260807 18:16:46.829380 9746 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 74b7e49ff11046dba46d21f00d9fd15a: Generated new TSK 1
I20260807 18:16:46.829843 9746 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 18:16:47.090031 9737 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:16:47.090330 9737 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:16:47.090477 9737 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:16:47.098057 9737 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:16:47.098176 9737 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:16:47.098343 9737 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.154.1
I20260807 18:16:47.108713 9737 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.154.20:44143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.1.keytab
--principal=kudu/127.18.154.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.154.1:34909
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.154.1
--webserver_port=40111
--tserver_master_addrs=127.18.154.62:46089
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.154.1
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:16:47.110214 9737 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:16:47.111711 9737 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:16:47.127130 9778 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:16:47.127108 9777 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:16:47.128535 9780 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:16:47.129467 9737 server_base.cc:1034] running on GCE node
I20260807 18:16:47.130242 9737 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:16:47.131323 9737 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:16:47.132596 9737 hybrid_clock.cc:648] HybridClock initialized: now 1786126607132532 us; error 52 us; skew 500 ppm
I20260807 18:16:47.132957 9737 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:16:47 dist-test-slave-gkw7 krb5kdc[8868](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126607, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:16:47.136521 9737 init.cc:377] Logged in from keytab as kudu/127.18.154.1@KRBTEST.COM (short username kudu)
I20260807 18:16:47.139307 9737 webserver.cc:460] Webserver started at http://127.18.154.1:40111/ using document root <none> and password file <none>
I20260807 18:16:47.140080 9737 fs_manager.cc:359] Metadata directory not provided
I20260807 18:16:47.140244 9737 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:16:47.146512 9737 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 18:16:47.149585 9787 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:16:47.151154 9737 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 18:16:47.151352 9737 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/ts-0/wal
uuid: "b3f50a9f91964b79b66d73c38ac9ab47"
format_stamp: "Formatted at 2026-08-07 18:16:24 on dist-test-slave-gkw7"
server_key: "9bcaaa5a6d0b685de9523261f10c7145"
server_key_iv: "6e711d227088d40fdf26e493e31bed16"
server_key_version: "encryptionkey@0"
I20260807 18:16:47.152464 9737 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:16:47.170293 9737 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:16:47.174851 9737 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:16:47.175158 9737 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:16:47.176208 9737 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:16:47.177626 9737 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:16:47.177742 9737 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:16:47.177881 9737 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:16:47.177983 9737 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:16:47.211354 9737 rpc_server.cc:307] RPC server started. Bound to: 127.18.154.1:34909
I20260807 18:16:47.211439 9935 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.154.1:34909 every 8 connection(s)
I20260807 18:16:47.212891 9737 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/ts-0/data/info.pb
I20260807 18:16:47.213872 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 9737
I20260807 18:16:47.214829 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.154.2:42929
--local_ip_for_outbound_sockets=127.18.154.2
--tserver_master_addrs=127.18.154.62:46089
--webserver_port=44051
--webserver_interface=127.18.154.2
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.2.keytab
--principal=kudu/127.18.154.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.154.20:44143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 18:16:47 dist-test-slave-gkw7 krb5kdc[8868](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126607, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.1@KRBTEST.COM for kudu/127.18.154.62@KRBTEST.COM
I20260807 18:16:47.243248 9941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:47.216682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:46089 (local address 127.18.154.1:38547)
0807 18:16:47.217477 (+ 795us) negotiation.cc:107] Waiting for socket to connect
0807 18:16:47.217523 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 18:16:47.219879 (+ 2356us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:16:47.221645 (+ 1766us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:16:47.221657 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:16:47.222395 (+ 738us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:16:47.223095 (+ 700us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:16:47.223117 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:16:47.225893 (+ 2776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:16:47.225901 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:16:47.227904 (+ 2003us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:16:47.227921 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:16:47.229418 (+ 1497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:16:47.229428 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:16:47.229796 (+ 368us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:16:47.230657 (+ 861us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:16:47.230693 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:16:47.233526 (+ 2833us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:16:47.238751 (+ 5225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:16:47.238759 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:16:47.238772 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:16:47.239244 (+ 472us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:16:47.239789 (+ 545us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:16:47.239792 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:16:47.239794 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:16:47.239981 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:16:47.241021 (+ 1040us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:16:47.241027 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:16:47.241347 (+ 320us) client_negotiation.cc:769] Sending connection context
0807 18:16:47.241665 (+ 318us) client_negotiation.cc:240] Negotiation successful
0807 18:16:47.242100 (+ 435us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":544,"thread_start_us":335,"threads_started":1}
I20260807 18:16:47.243793 9939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:47.216301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.1:38547 (local address 127.18.154.62:46089)
0807 18:16:47.216958 (+ 657us) server_negotiation.cc:206] Beginning negotiation
0807 18:16:47.216970 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 18:16:47.217744 (+ 774us) server_negotiation.cc:407] Connection header received
0807 18:16:47.220509 (+ 2765us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:16:47.220533 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:16:47.221030 (+ 497us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:16:47.221471 (+ 441us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:16:47.223348 (+ 1877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:16:47.225699 (+ 2351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:16:47.228088 (+ 2389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:16:47.229160 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:16:47.229313 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:16:47.233901 (+ 4588us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:16:47.233915 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:16:47.233939 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:16:47.234000 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:16:47.238581 (+ 4581us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:16:47.239422 (+ 841us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:16:47.239434 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:16:47.239445 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:16:47.239666 (+ 221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:16:47.240106 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:16:47.240114 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:16:47.240119 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:16:47.240872 (+ 753us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:16:47.241097 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0807 18:16:47.241961 (+ 864us) server_negotiation.cc:299] Negotiation successful
0807 18:16:47.242445 (+ 484us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":445,"thread_start_us":301,"threads_started":1}
I20260807 18:16:47.246642 9936 heartbeater.cc:347] Connected to a master server at 127.18.154.62:46089
I20260807 18:16:47.247020 9936 heartbeater.cc:464] Registering TS with master...
I20260807 18:16:47.248052 9936 heartbeater.cc:511] Master 127.18.154.62:46089 requested a full tablet report, sending...
I20260807 18:16:47.250730 9679 ts_manager.cc:194] Registered new tserver with Master: b3f50a9f91964b79b66d73c38ac9ab47 (127.18.154.1:34909)
I20260807 18:16:47.252373 9679 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.154.1@KRBTEST.COM'} at 127.18.154.1:38547
W20260807 18:16:47.579439 9942 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:16:47.579744 9942 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:16:47.579906 9942 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:16:47.587261 9942 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:16:47.587384 9942 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:16:47.587608 9942 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.154.2
I20260807 18:16:47.597445 9942 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.154.20:44143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.2.keytab
--principal=kudu/127.18.154.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.154.2:42929
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.154.2
--webserver_port=44051
--tserver_master_addrs=127.18.154.62:46089
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.154.2
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:16:47.598907 9942 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:16:47.600313 9942 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:16:47.614938 9951 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:16:47.614948 9955 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:16:47.615662 9948 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:16:47.616019 9942 server_base.cc:1034] running on GCE node
I20260807 18:16:47.616611 9942 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:16:47.617664 9942 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:16:47.618911 9942 hybrid_clock.cc:648] HybridClock initialized: now 1786126607618842 us; error 63 us; skew 500 ppm
I20260807 18:16:47.619272 9942 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:16:47 dist-test-slave-gkw7 krb5kdc[8868](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126607, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:16:47.622678 9942 init.cc:377] Logged in from keytab as kudu/127.18.154.2@KRBTEST.COM (short username kudu)
I20260807 18:16:47.624238 9942 webserver.cc:460] Webserver started at http://127.18.154.2:44051/ using document root <none> and password file <none>
I20260807 18:16:47.624969 9942 fs_manager.cc:359] Metadata directory not provided
I20260807 18:16:47.625135 9942 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:16:47.630877 9942 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260807 18:16:47.633744 9967 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:16:47.635051 9942 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 18:16:47.635237 9942 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/ts-1/wal
uuid: "3487895bdebd4925b1e0ee7b961c18d9"
format_stamp: "Formatted at 2026-08-07 18:16:24 on dist-test-slave-gkw7"
server_key: "5479e736d511ce1d944ebffc9f018b16"
server_key_iv: "3268433d2eac4524a60aef8bd486c087"
server_key_version: "encryptionkey@0"
I20260807 18:16:47.636231 9942 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:16:47.660013 9942 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:16:47.664256 9942 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:16:47.664547 9942 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:16:47.665649 9942 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:16:47.667001 9942 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:16:47.667135 9942 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:16:47.667271 9942 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:16:47.667361 9942 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:16:47.701345 9942 rpc_server.cc:307] RPC server started. Bound to: 127.18.154.2:42929
I20260807 18:16:47.701395 10125 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.154.2:42929 every 8 connection(s)
I20260807 18:16:47.703249 9942 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/ts-1/data/info.pb
I20260807 18:16:47.712922 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 9942
I20260807 18:16:47.713641 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.154.3:33869
--local_ip_for_outbound_sockets=127.18.154.3
--tserver_master_addrs=127.18.154.62:46089
--webserver_port=43187
--webserver_interface=127.18.154.3
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.3.keytab
--principal=kudu/127.18.154.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.154.20:44143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 18:16:47 dist-test-slave-gkw7 krb5kdc[8868](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126607, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.2@KRBTEST.COM for kudu/127.18.154.62@KRBTEST.COM
I20260807 18:16:47.726224 9939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:47.706387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.2:48455 (local address 127.18.154.62:46089)
0807 18:16:47.706581 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0807 18:16:47.706586 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:16:47.707899 (+ 1313us) server_negotiation.cc:407] Connection header received
0807 18:16:47.709992 (+ 2093us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:16:47.709998 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:16:47.710120 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:16:47.710241 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:16:47.711775 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:16:47.712558 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:16:47.714488 (+ 1930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:16:47.715267 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:16:47.715416 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:16:47.718930 (+ 3514us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:16:47.718939 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:16:47.718947 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:16:47.718997 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:16:47.722829 (+ 3832us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:16:47.723629 (+ 800us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:16:47.723637 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:16:47.723643 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:16:47.723813 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:16:47.724233 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:16:47.724241 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:16:47.724247 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:16:47.724676 (+ 429us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:16:47.724839 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0807 18:16:47.725598 (+ 759us) server_negotiation.cc:299] Negotiation successful
0807 18:16:47.725868 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260807 18:16:47.727252 10129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:47.706865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:46089 (local address 127.18.154.2:48455)
0807 18:16:47.707729 (+ 864us) negotiation.cc:107] Waiting for socket to connect
0807 18:16:47.707768 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 18:16:47.709704 (+ 1936us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:16:47.710493 (+ 789us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:16:47.710502 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:16:47.711016 (+ 514us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:16:47.711592 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:16:47.711610 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:16:47.712836 (+ 1226us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:16:47.712840 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:16:47.714333 (+ 1493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:16:47.714346 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:16:47.715488 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:16:47.715493 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:16:47.715689 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:16:47.716386 (+ 697us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:16:47.716415 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:16:47.718681 (+ 2266us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:16:47.722982 (+ 4301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:16:47.722990 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:16:47.723004 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:16:47.723483 (+ 479us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:16:47.723924 (+ 441us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:16:47.723927 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:16:47.723930 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:16:47.724115 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:16:47.724798 (+ 683us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:16:47.724804 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:16:47.725145 (+ 341us) client_negotiation.cc:769] Sending connection context
0807 18:16:47.725475 (+ 330us) client_negotiation.cc:240] Negotiation successful
0807 18:16:47.726071 (+ 596us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":595,"thread_start_us":388,"threads_started":1}
I20260807 18:16:47.728816 10126 heartbeater.cc:347] Connected to a master server at 127.18.154.62:46089
I20260807 18:16:47.729190 10126 heartbeater.cc:464] Registering TS with master...
I20260807 18:16:47.730181 10126 heartbeater.cc:511] Master 127.18.154.62:46089 requested a full tablet report, sending...
I20260807 18:16:47.731922 9679 ts_manager.cc:194] Registered new tserver with Master: 3487895bdebd4925b1e0ee7b961c18d9 (127.18.154.2:42929)
I20260807 18:16:47.733284 9679 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.154.2@KRBTEST.COM'} at 127.18.154.2:48455
W20260807 18:16:48.073866 10131 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:16:48.074172 10131 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:16:48.074270 10131 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:16:48.081738 10131 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:16:48.081888 10131 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:16:48.082005 10131 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.154.3
I20260807 18:16:48.092011 10131 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.154.20:44143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.3.keytab
--principal=kudu/127.18.154.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.154.3:33869
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.154.3
--webserver_port=43187
--tserver_master_addrs=127.18.154.62:46089
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.154.3
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:16:48.093343 10131 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:16:48.094729 10131 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:16:48.108817 10140 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:16:48.108853 10143 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:16:48.108762 10139 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:16:48.109895 10131 server_base.cc:1034] running on GCE node
I20260807 18:16:48.110671 10131 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:16:48.111761 10131 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:16:48.113029 10131 hybrid_clock.cc:648] HybridClock initialized: now 1786126608112965 us; error 51 us; skew 500 ppm
I20260807 18:16:48.113405 10131 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:16:48 dist-test-slave-gkw7 krb5kdc[8868](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126608, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:16:48.116986 10131 init.cc:377] Logged in from keytab as kudu/127.18.154.3@KRBTEST.COM (short username kudu)
I20260807 18:16:48.118883 10131 webserver.cc:460] Webserver started at http://127.18.154.3:43187/ using document root <none> and password file <none>
I20260807 18:16:48.119709 10131 fs_manager.cc:359] Metadata directory not provided
I20260807 18:16:48.119887 10131 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:16:48.125862 10131 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260807 18:16:48.128816 10151 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:16:48.130340 10131 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:16:48.130527 10131 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/ts-2/wal
uuid: "90873f7a558149f094dd37e015a2d8f8"
format_stamp: "Formatted at 2026-08-07 18:16:25 on dist-test-slave-gkw7"
server_key: "11c2a1e3e5528eb5d85d9fe5a4b412a4"
server_key_iv: "9781bbb23881a1c542e42a6b05af0077"
server_key_version: "encryptionkey@0"
I20260807 18:16:48.131536 10131 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:16:48.155617 10131 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:16:48.160677 10131 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:16:48.160987 10131 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:16:48.162227 10131 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:16:48.163750 10131 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:16:48.163882 10131 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:16:48.164028 10131 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:16:48.164120 10131 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:16:48.200428 10131 rpc_server.cc:307] RPC server started. Bound to: 127.18.154.3:33869
I20260807 18:16:48.200521 10311 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.154.3:33869 every 8 connection(s)
I20260807 18:16:48.202023 10131 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/ts-2/data/info.pb
I20260807 18:16:48.210098 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 10131
Aug 07 18:16:48 dist-test-slave-gkw7 krb5kdc[8868](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126608, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.3@KRBTEST.COM for kudu/127.18.154.62@KRBTEST.COM
I20260807 18:16:48.229707 9939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:48.204974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.3:53771 (local address 127.18.154.62:46089)
0807 18:16:48.205270 (+ 296us) server_negotiation.cc:206] Beginning negotiation
0807 18:16:48.205275 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:16:48.206508 (+ 1233us) server_negotiation.cc:407] Connection header received
0807 18:16:48.209140 (+ 2632us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:16:48.209145 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:16:48.209262 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:16:48.209383 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:16:48.211695 (+ 2312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:16:48.213185 (+ 1490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:16:48.216683 (+ 3498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:16:48.217688 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:16:48.217833 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:16:48.222462 (+ 4629us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:16:48.222468 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:16:48.222473 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:16:48.222518 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:16:48.226236 (+ 3718us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:16:48.226972 (+ 736us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:16:48.226979 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:16:48.226985 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:16:48.227125 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:16:48.227635 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:16:48.227638 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:16:48.227640 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:16:48.228011 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:16:48.228160 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0807 18:16:48.229020 (+ 860us) server_negotiation.cc:299] Negotiation successful
0807 18:16:48.229268 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260807 18:16:48.230647 10315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:48.205459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:46089 (local address 127.18.154.3:53771)
0807 18:16:48.206337 (+ 878us) negotiation.cc:107] Waiting for socket to connect
0807 18:16:48.206378 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 18:16:48.208854 (+ 2476us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:16:48.209705 (+ 851us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:16:48.209719 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:16:48.210516 (+ 797us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:16:48.211431 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:16:48.211466 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:16:48.213479 (+ 2013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:16:48.213487 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:16:48.216456 (+ 2969us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:16:48.216478 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:16:48.218331 (+ 1853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:16:48.218341 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:16:48.218762 (+ 421us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:16:48.219671 (+ 909us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:16:48.219702 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:16:48.222201 (+ 2499us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:16:48.226384 (+ 4183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:16:48.226391 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:16:48.226406 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:16:48.226841 (+ 435us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:16:48.227236 (+ 395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:16:48.227242 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:16:48.227247 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:16:48.227502 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:16:48.228176 (+ 674us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:16:48.228184 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:16:48.228615 (+ 431us) client_negotiation.cc:769] Sending connection context
0807 18:16:48.228851 (+ 236us) client_negotiation.cc:240] Negotiation successful
0807 18:16:48.229272 (+ 421us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":612,"thread_start_us":351,"threads_started":1}
I20260807 18:16:48.232229 10313 heartbeater.cc:347] Connected to a master server at 127.18.154.62:46089
I20260807 18:16:48.232597 10313 heartbeater.cc:464] Registering TS with master...
I20260807 18:16:48.233677 10313 heartbeater.cc:511] Master 127.18.154.62:46089 requested a full tablet report, sending...
I20260807 18:16:48.235420 9679 ts_manager.cc:194] Registered new tserver with Master: 90873f7a558149f094dd37e015a2d8f8 (127.18.154.3:33869)
I20260807 18:16:48.236418 9679 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.154.3@KRBTEST.COM'} at 127.18.154.3:53771
I20260807 18:16:48.237824 10316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:48.211143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56782 (local address 127.18.154.62:46089)
0807 18:16:48.220946 (+ 9803us) server_negotiation.cc:206] Beginning negotiation
0807 18:16:48.220952 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:16:48.221568 (+ 616us) server_negotiation.cc:407] Connection header received
0807 18:16:48.221864 (+ 296us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:16:48.221872 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:16:48.221940 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:16:48.222058 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:16:48.224356 (+ 2298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:16:48.225176 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:16:48.227020 (+ 1844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:16:48.227672 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:16:48.227773 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:16:48.231187 (+ 3414us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:16:48.231191 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:16:48.231195 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:16:48.231226 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:16:48.234656 (+ 3430us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:16:48.235436 (+ 780us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:16:48.235442 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:16:48.235446 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:16:48.235629 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:16:48.236175 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:16:48.236180 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:16:48.236183 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:16:48.236503 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:16:48.236794 (+ 291us) server_negotiation.cc:1035] Waiting for connection context
0807 18:16:48.237281 (+ 487us) server_negotiation.cc:299] Negotiation successful
0807 18:16:48.237477 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9622,"thread_start_us":8372,"threads_started":1}
I20260807 18:16:48.242318 19048 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 18:16:48.249130 10316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:48.243456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56786 (local address 127.18.154.62:46089)
0807 18:16:48.243681 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0807 18:16:48.243685 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:16:48.244092 (+ 407us) server_negotiation.cc:407] Connection header received
0807 18:16:48.244260 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:16:48.244265 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:16:48.244400 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:16:48.244534 (+ 134us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:16:48.244932 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:16:48.245769 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:16:48.247300 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:16:48.247736 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:16:48.247832 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:16:48.248304 (+ 472us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:16:48.248602 (+ 298us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0807 18:16:48.248908 (+ 306us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:56786: token expired
Metrics: {"server-negotiator.queue_time_us":76}
W20260807 18:16:48.249279 10316 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:56786: token expired
W20260807 18:16:48.250028 9576 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.18.154.62:46089)
I20260807 18:16:48.256022 9936 heartbeater.cc:503] Master 127.18.154.62:46089 was elected leader, sending a full tablet report...
I20260807 18:16:48.264303 10316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:48.250605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56792 (local address 127.18.154.62:46089)
0807 18:16:48.250772 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0807 18:16:48.250776 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:16:48.250824 (+ 48us) server_negotiation.cc:407] Connection header received
0807 18:16:48.250953 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:16:48.250956 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:16:48.251014 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:16:48.251094 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:16:48.252576 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:16:48.253359 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:16:48.254838 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:16:48.255242 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:16:48.255342 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:16:48.258668 (+ 3326us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:16:48.258674 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:16:48.258678 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:16:48.258723 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:16:48.261415 (+ 2692us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:16:48.262379 (+ 964us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:16:48.262383 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:16:48.262385 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:16:48.262515 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:16:48.262946 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:16:48.262949 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:16:48.262951 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:16:48.263269 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:16:48.263449 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0807 18:16:48.263789 (+ 340us) server_negotiation.cc:299] Negotiation successful
0807 18:16:48.263963 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 18:16:48.267967 9679 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56792:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 18:16:48.275588 9679 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:16:48.312412 10327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:48.288078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.1:34909 (local address 127.0.0.1:34626)
0807 18:16:48.300194 (+ 12116us) negotiation.cc:107] Waiting for socket to connect
0807 18:16:48.300252 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0807 18:16:48.300529 (+ 277us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:16:48.303327 (+ 2798us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:16:48.303340 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:16:48.303389 (+ 49us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:16:48.304027 (+ 638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:16:48.304044 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:16:48.307373 (+ 3329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:16:48.307381 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:16:48.309716 (+ 2335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:16:48.309729 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:16:48.311373 (+ 1644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:16:48.311379 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:16:48.311588 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:16:48.311650 (+ 62us) client_negotiation.cc:769] Sending connection context
0807 18:16:48.311810 (+ 160us) client_negotiation.cc:240] Negotiation successful
0807 18:16:48.311997 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11758,"thread_start_us":16355,"threads_started":1}
I20260807 18:16:48.313251 10328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:48.288462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34626 (local address 127.18.154.1:34909)
0807 18:16:48.301265 (+ 12803us) server_negotiation.cc:206] Beginning negotiation
0807 18:16:48.301274 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:16:48.301305 (+ 31us) server_negotiation.cc:407] Connection header received
0807 18:16:48.301415 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:16:48.301424 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:16:48.301852 (+ 428us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:16:48.304338 (+ 2486us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:16:48.304630 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:16:48.307235 (+ 2605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:16:48.309874 (+ 2639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:16:48.311114 (+ 1240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:16:48.311280 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:16:48.311532 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0807 18:16:48.312709 (+ 1177us) server_negotiation.cc:299] Negotiation successful
0807 18:16:48.312900 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12627,"thread_start_us":12482,"threads_started":1}
I20260807 18:16:48.314631 10326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:48.288328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.3:33869 (local address 127.0.0.1:32978)
0807 18:16:48.304716 (+ 16388us) negotiation.cc:107] Waiting for socket to connect
0807 18:16:48.304739 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 18:16:48.304886 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:16:48.306324 (+ 1438us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:16:48.306329 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:16:48.306357 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:16:48.306689 (+ 332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:16:48.306700 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:16:48.308802 (+ 2102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:16:48.308809 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:16:48.310874 (+ 2065us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:16:48.310886 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:16:48.313637 (+ 2751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:16:48.313644 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:16:48.313823 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:16:48.313857 (+ 34us) client_negotiation.cc:769] Sending connection context
0807 18:16:48.313954 (+ 97us) client_negotiation.cc:240] Negotiation successful
0807 18:16:48.314072 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16248,"thread_start_us":11269,"threads_started":1}
I20260807 18:16:48.314771 10329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:48.288931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32978 (local address 127.18.154.3:33869)
0807 18:16:48.299116 (+ 10185us) server_negotiation.cc:206] Beginning negotiation
0807 18:16:48.299131 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 18:16:48.304839 (+ 5708us) server_negotiation.cc:407] Connection header received
0807 18:16:48.305045 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:16:48.305058 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:16:48.305380 (+ 322us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:16:48.305990 (+ 610us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:16:48.306907 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:16:48.308604 (+ 1697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:16:48.311040 (+ 2436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:16:48.312514 (+ 1474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:16:48.312648 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:16:48.312878 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0807 18:16:48.314127 (+ 1249us) server_negotiation.cc:299] Negotiation successful
0807 18:16:48.314345 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9900,"thread_start_us":10219,"threads_started":1}
I20260807 18:16:48.315307 10331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:48.301331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.2:42929 (local address 127.0.0.1:43520)
0807 18:16:48.306063 (+ 4732us) negotiation.cc:107] Waiting for socket to connect
0807 18:16:48.306087 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 18:16:48.306238 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:16:48.307430 (+ 1192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:16:48.307436 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:16:48.307470 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:16:48.307916 (+ 446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:16:48.307927 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:16:48.310560 (+ 2633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:16:48.310567 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:16:48.312445 (+ 1878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:16:48.312458 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:16:48.314450 (+ 1992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:16:48.314460 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:16:48.314680 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:16:48.314720 (+ 40us) client_negotiation.cc:769] Sending connection context
0807 18:16:48.314805 (+ 85us) client_negotiation.cc:240] Negotiation successful
0807 18:16:48.314962 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4583,"thread_start_us":4401,"threads_started":1}
I20260807 18:16:48.316411 10330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:48.301117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43520 (local address 127.18.154.2:42929)
0807 18:16:48.306770 (+ 5653us) server_negotiation.cc:206] Beginning negotiation
0807 18:16:48.306786 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 18:16:48.306840 (+ 54us) server_negotiation.cc:407] Connection header received
0807 18:16:48.306933 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:16:48.306940 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:16:48.307185 (+ 245us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:16:48.308119 (+ 934us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:16:48.308597 (+ 478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:16:48.310406 (+ 1809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:16:48.312676 (+ 2270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:16:48.314139 (+ 1463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:16:48.315719 (+ 1580us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:16:48.315871 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0807 18:16:48.315925 (+ 54us) server_negotiation.cc:299] Negotiation successful
0807 18:16:48.316045 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5501,"thread_start_us":5389,"threads_started":1}
I20260807 18:16:48.317997 9847 tablet_service.cc:1467] Processing CreateTablet for tablet 97473341374741d6b3a437769d5d74a2 (DEFAULT_TABLE table=test-table [id=2365d5179ea247bb8ddb601ff096752b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:16:48.319175 10221 tablet_service.cc:1467] Processing CreateTablet for tablet 97473341374741d6b3a437769d5d74a2 (DEFAULT_TABLE table=test-table [id=2365d5179ea247bb8ddb601ff096752b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:16:48.319751 9847 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 97473341374741d6b3a437769d5d74a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:16:48.320457 10221 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 97473341374741d6b3a437769d5d74a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:16:48.320055 10035 tablet_service.cc:1467] Processing CreateTablet for tablet 97473341374741d6b3a437769d5d74a2 (DEFAULT_TABLE table=test-table [id=2365d5179ea247bb8ddb601ff096752b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:16:48.321354 10035 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 97473341374741d6b3a437769d5d74a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:16:48.334026 10332 tablet_bootstrap.cc:492] T 97473341374741d6b3a437769d5d74a2 P 90873f7a558149f094dd37e015a2d8f8: Bootstrap starting.
I20260807 18:16:48.336030 10333 tablet_bootstrap.cc:492] T 97473341374741d6b3a437769d5d74a2 P b3f50a9f91964b79b66d73c38ac9ab47: Bootstrap starting.
I20260807 18:16:48.337582 10336 tablet_bootstrap.cc:492] T 97473341374741d6b3a437769d5d74a2 P 3487895bdebd4925b1e0ee7b961c18d9: Bootstrap starting.
I20260807 18:16:48.340353 10332 tablet_bootstrap.cc:654] T 97473341374741d6b3a437769d5d74a2 P 90873f7a558149f094dd37e015a2d8f8: Neither blocks nor log segments found. Creating new log.
I20260807 18:16:48.340350 10333 tablet_bootstrap.cc:654] T 97473341374741d6b3a437769d5d74a2 P b3f50a9f91964b79b66d73c38ac9ab47: Neither blocks nor log segments found. Creating new log.
I20260807 18:16:48.340683 10336 tablet_bootstrap.cc:654] T 97473341374741d6b3a437769d5d74a2 P 3487895bdebd4925b1e0ee7b961c18d9: Neither blocks nor log segments found. Creating new log.
I20260807 18:16:48.341837 10336 log.cc:824] T 97473341374741d6b3a437769d5d74a2 P 3487895bdebd4925b1e0ee7b961c18d9: Log is configured to *not* fsync() on all Append() calls
I20260807 18:16:48.341900 10332 log.cc:824] T 97473341374741d6b3a437769d5d74a2 P 90873f7a558149f094dd37e015a2d8f8: Log is configured to *not* fsync() on all Append() calls
I20260807 18:16:48.341899 10333 log.cc:824] T 97473341374741d6b3a437769d5d74a2 P b3f50a9f91964b79b66d73c38ac9ab47: Log is configured to *not* fsync() on all Append() calls
I20260807 18:16:48.344551 10336 tablet_bootstrap.cc:492] T 97473341374741d6b3a437769d5d74a2 P 3487895bdebd4925b1e0ee7b961c18d9: No bootstrap required, opened a new log
I20260807 18:16:48.344879 10336 ts_tablet_manager.cc:1397] T 97473341374741d6b3a437769d5d74a2 P 3487895bdebd4925b1e0ee7b961c18d9: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.003s
I20260807 18:16:48.347873 10333 tablet_bootstrap.cc:492] T 97473341374741d6b3a437769d5d74a2 P b3f50a9f91964b79b66d73c38ac9ab47: No bootstrap required, opened a new log
I20260807 18:16:48.347986 10332 tablet_bootstrap.cc:492] T 97473341374741d6b3a437769d5d74a2 P 90873f7a558149f094dd37e015a2d8f8: No bootstrap required, opened a new log
I20260807 18:16:48.348146 10333 ts_tablet_manager.cc:1397] T 97473341374741d6b3a437769d5d74a2 P b3f50a9f91964b79b66d73c38ac9ab47: Time spent bootstrapping tablet: real 0.012s user 0.004s sys 0.005s
I20260807 18:16:48.348347 10332 ts_tablet_manager.cc:1397] T 97473341374741d6b3a437769d5d74a2 P 90873f7a558149f094dd37e015a2d8f8: Time spent bootstrapping tablet: real 0.015s user 0.007s sys 0.004s
I20260807 18:16:48.356165 10332 raft_consensus.cc:348] T 97473341374741d6b3a437769d5d74a2 P 90873f7a558149f094dd37e015a2d8f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3f50a9f91964b79b66d73c38ac9ab47" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 34909 } } peers { permanent_uuid: "90873f7a558149f094dd37e015a2d8f8" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 33869 } } peers { permanent_uuid: "3487895bdebd4925b1e0ee7b961c18d9" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 42929 } }
I20260807 18:16:48.356165 10336 raft_consensus.cc:348] T 97473341374741d6b3a437769d5d74a2 P 3487895bdebd4925b1e0ee7b961c18d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3f50a9f91964b79b66d73c38ac9ab47" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 34909 } } peers { permanent_uuid: "90873f7a558149f094dd37e015a2d8f8" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 33869 } } peers { permanent_uuid: "3487895bdebd4925b1e0ee7b961c18d9" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 42929 } }
I20260807 18:16:48.356732 10332 raft_consensus.cc:374] T 97473341374741d6b3a437769d5d74a2 P 90873f7a558149f094dd37e015a2d8f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:16:48.356832 10336 raft_consensus.cc:374] T 97473341374741d6b3a437769d5d74a2 P 3487895bdebd4925b1e0ee7b961c18d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:16:48.356873 10332 raft_consensus.cc:729] T 97473341374741d6b3a437769d5d74a2 P 90873f7a558149f094dd37e015a2d8f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90873f7a558149f094dd37e015a2d8f8, State: Initialized, Role: FOLLOWER
I20260807 18:16:48.356982 10336 raft_consensus.cc:729] T 97473341374741d6b3a437769d5d74a2 P 3487895bdebd4925b1e0ee7b961c18d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3487895bdebd4925b1e0ee7b961c18d9, State: Initialized, Role: FOLLOWER
I20260807 18:16:48.357792 10332 consensus_queue.cc:260] T 97473341374741d6b3a437769d5d74a2 P 90873f7a558149f094dd37e015a2d8f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3f50a9f91964b79b66d73c38ac9ab47" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 34909 } } peers { permanent_uuid: "90873f7a558149f094dd37e015a2d8f8" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 33869 } } peers { permanent_uuid: "3487895bdebd4925b1e0ee7b961c18d9" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 42929 } }
I20260807 18:16:48.357826 10336 consensus_queue.cc:260] T 97473341374741d6b3a437769d5d74a2 P 3487895bdebd4925b1e0ee7b961c18d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3f50a9f91964b79b66d73c38ac9ab47" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 34909 } } peers { permanent_uuid: "90873f7a558149f094dd37e015a2d8f8" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 33869 } } peers { permanent_uuid: "3487895bdebd4925b1e0ee7b961c18d9" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 42929 } }
I20260807 18:16:48.358966 10333 raft_consensus.cc:348] T 97473341374741d6b3a437769d5d74a2 P b3f50a9f91964b79b66d73c38ac9ab47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3f50a9f91964b79b66d73c38ac9ab47" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 34909 } } peers { permanent_uuid: "90873f7a558149f094dd37e015a2d8f8" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 33869 } } peers { permanent_uuid: "3487895bdebd4925b1e0ee7b961c18d9" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 42929 } }
I20260807 18:16:48.359282 10126 heartbeater.cc:503] Master 127.18.154.62:46089 was elected leader, sending a full tablet report...
I20260807 18:16:48.359436 10333 raft_consensus.cc:374] T 97473341374741d6b3a437769d5d74a2 P b3f50a9f91964b79b66d73c38ac9ab47 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:16:48.359531 10333 raft_consensus.cc:729] T 97473341374741d6b3a437769d5d74a2 P b3f50a9f91964b79b66d73c38ac9ab47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3f50a9f91964b79b66d73c38ac9ab47, State: Initialized, Role: FOLLOWER
I20260807 18:16:48.360064 10336 ts_tablet_manager.cc:1428] T 97473341374741d6b3a437769d5d74a2 P 3487895bdebd4925b1e0ee7b961c18d9: Time spent starting tablet: real 0.015s user 0.011s sys 0.004s
I20260807 18:16:48.360206 10333 consensus_queue.cc:260] T 97473341374741d6b3a437769d5d74a2 P b3f50a9f91964b79b66d73c38ac9ab47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3f50a9f91964b79b66d73c38ac9ab47" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 34909 } } peers { permanent_uuid: "90873f7a558149f094dd37e015a2d8f8" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 33869 } } peers { permanent_uuid: "3487895bdebd4925b1e0ee7b961c18d9" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 42929 } }
I20260807 18:16:48.362103 10333 ts_tablet_manager.cc:1428] T 97473341374741d6b3a437769d5d74a2 P b3f50a9f91964b79b66d73c38ac9ab47: Time spent starting tablet: real 0.014s user 0.015s sys 0.001s
I20260807 18:16:48.363364 10313 heartbeater.cc:503] Master 127.18.154.62:46089 was elected leader, sending a full tablet report...
I20260807 18:16:48.364171 10332 ts_tablet_manager.cc:1428] T 97473341374741d6b3a437769d5d74a2 P 90873f7a558149f094dd37e015a2d8f8: Time spent starting tablet: real 0.016s user 0.014s sys 0.000s
I20260807 18:16:48.449353 10342 raft_consensus.cc:482] T 97473341374741d6b3a437769d5d74a2 P 3487895bdebd4925b1e0ee7b961c18d9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:16:48.449827 10342 raft_consensus.cc:504] T 97473341374741d6b3a437769d5d74a2 P 3487895bdebd4925b1e0ee7b961c18d9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3f50a9f91964b79b66d73c38ac9ab47" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 34909 } } peers { permanent_uuid: "90873f7a558149f094dd37e015a2d8f8" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 33869 } } peers { permanent_uuid: "3487895bdebd4925b1e0ee7b961c18d9" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 42929 } }
I20260807 18:16:48.451756 10342 leader_election.cc:288] T 97473341374741d6b3a437769d5d74a2 P 3487895bdebd4925b1e0ee7b961c18d9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b3f50a9f91964b79b66d73c38ac9ab47 (127.18.154.1:34909), 90873f7a558149f094dd37e015a2d8f8 (127.18.154.3:33869)
W20260807 18:16:48.453900 10314 tablet.cc:2369] T 97473341374741d6b3a437769d5d74a2 P 90873f7a558149f094dd37e015a2d8f8: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 18:16:48.455570 10128 tablet.cc:2369] T 97473341374741d6b3a437769d5d74a2 P 3487895bdebd4925b1e0ee7b961c18d9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:16:48.460356 10328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:48.452432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.2:40779 (local address 127.18.154.1:34909)
0807 18:16:48.452668 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0807 18:16:48.452675 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:16:48.453613 (+ 938us) server_negotiation.cc:407] Connection header received
0807 18:16:48.453882 (+ 269us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:16:48.453889 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:16:48.453988 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:16:48.454110 (+ 122us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:16:48.454952 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:16:48.455969 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:16:48.458271 (+ 2302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:16:48.459407 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:16:48.459574 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:16:48.459671 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0807 18:16:48.459908 (+ 237us) server_negotiation.cc:299] Negotiation successful
0807 18:16:48.460053 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260807 18:16:48.460518 10345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:48.452153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.1:34909 (local address 127.18.154.2:40779)
0807 18:16:48.453475 (+ 1322us) negotiation.cc:107] Waiting for socket to connect
0807 18:16:48.453495 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 18:16:48.453707 (+ 212us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:16:48.454268 (+ 561us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:16:48.454273 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:16:48.454305 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:16:48.454726 (+ 421us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:16:48.454745 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:16:48.456130 (+ 1385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:16:48.456136 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:16:48.458086 (+ 1950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:16:48.458107 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:16:48.459610 (+ 1503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:16:48.459614 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:16:48.459743 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:16:48.459768 (+ 25us) client_negotiation.cc:769] Sending connection context
0807 18:16:48.459849 (+ 81us) client_negotiation.cc:240] Negotiation successful
0807 18:16:48.459970 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1191,"mutex_wait_us":21,"thread_start_us":790,"threads_started":1}
I20260807 18:16:48.460999 10329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:48.452445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.2:43749 (local address 127.18.154.3:33869)
0807 18:16:48.452667 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0807 18:16:48.452674 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:16:48.453614 (+ 940us) server_negotiation.cc:407] Connection header received
0807 18:16:48.454340 (+ 726us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:16:48.454345 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:16:48.454431 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:16:48.454533 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:16:48.455228 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:16:48.456403 (+ 1175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:16:48.458595 (+ 2192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:16:48.459875 (+ 1280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:16:48.460035 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:16:48.460112 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0807 18:16:48.460587 (+ 475us) server_negotiation.cc:299] Negotiation successful
0807 18:16:48.460750 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260807 18:16:48.461838 10346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:48.452108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.3:33869 (local address 127.18.154.2:43749)
0807 18:16:48.453393 (+ 1285us) negotiation.cc:107] Waiting for socket to connect
0807 18:16:48.453419 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0807 18:16:48.453686 (+ 267us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:16:48.454578 (+ 892us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:16:48.454583 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:16:48.454614 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:16:48.455065 (+ 451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:16:48.455075 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:16:48.456585 (+ 1510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:16:48.456590 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:16:48.458433 (+ 1843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:16:48.458446 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:16:48.460111 (+ 1665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:16:48.460119 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:16:48.460311 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:16:48.460351 (+ 40us) client_negotiation.cc:769] Sending connection context
0807 18:16:48.461276 (+ 925us) client_negotiation.cc:240] Negotiation successful
0807 18:16:48.461423 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1126,"mutex_wait_us":2,"thread_start_us":795,"threads_started":1}
I20260807 18:16:48.461954 9874 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97473341374741d6b3a437769d5d74a2" candidate_uuid: "3487895bdebd4925b1e0ee7b961c18d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3f50a9f91964b79b66d73c38ac9ab47" is_pre_election: true
I20260807 18:16:48.462641 9874 raft_consensus.cc:2454] T 97473341374741d6b3a437769d5d74a2 P b3f50a9f91964b79b66d73c38ac9ab47 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3487895bdebd4925b1e0ee7b961c18d9 in term 0.
I20260807 18:16:48.462957 10248 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97473341374741d6b3a437769d5d74a2" candidate_uuid: "3487895bdebd4925b1e0ee7b961c18d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90873f7a558149f094dd37e015a2d8f8" is_pre_election: true
I20260807 18:16:48.463379 9974 leader_election.cc:302] T 97473341374741d6b3a437769d5d74a2 P 3487895bdebd4925b1e0ee7b961c18d9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3487895bdebd4925b1e0ee7b961c18d9, b3f50a9f91964b79b66d73c38ac9ab47; no voters:
I20260807 18:16:48.463528 10248 raft_consensus.cc:2454] T 97473341374741d6b3a437769d5d74a2 P 90873f7a558149f094dd37e015a2d8f8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3487895bdebd4925b1e0ee7b961c18d9 in term 0.
I20260807 18:16:48.463804 10342 raft_consensus.cc:2781] T 97473341374741d6b3a437769d5d74a2 P 3487895bdebd4925b1e0ee7b961c18d9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:16:48.463917 10342 raft_consensus.cc:482] T 97473341374741d6b3a437769d5d74a2 P 3487895bdebd4925b1e0ee7b961c18d9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:16:48.463986 10342 raft_consensus.cc:3037] T 97473341374741d6b3a437769d5d74a2 P 3487895bdebd4925b1e0ee7b961c18d9 [term 0 FOLLOWER]: Advancing to term 1
W20260807 18:16:48.465665 9938 tablet.cc:2369] T 97473341374741d6b3a437769d5d74a2 P b3f50a9f91964b79b66d73c38ac9ab47: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:16:48.466348 10342 raft_consensus.cc:504] T 97473341374741d6b3a437769d5d74a2 P 3487895bdebd4925b1e0ee7b961c18d9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3f50a9f91964b79b66d73c38ac9ab47" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 34909 } } peers { permanent_uuid: "90873f7a558149f094dd37e015a2d8f8" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 33869 } } peers { permanent_uuid: "3487895bdebd4925b1e0ee7b961c18d9" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 42929 } }
I20260807 18:16:48.467015 10342 leader_election.cc:288] T 97473341374741d6b3a437769d5d74a2 P 3487895bdebd4925b1e0ee7b961c18d9 [CANDIDATE]: Term 1 election: Requested vote from peers b3f50a9f91964b79b66d73c38ac9ab47 (127.18.154.1:34909), 90873f7a558149f094dd37e015a2d8f8 (127.18.154.3:33869)
I20260807 18:16:48.467506 9874 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97473341374741d6b3a437769d5d74a2" candidate_uuid: "3487895bdebd4925b1e0ee7b961c18d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3f50a9f91964b79b66d73c38ac9ab47"
I20260807 18:16:48.467622 10248 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97473341374741d6b3a437769d5d74a2" candidate_uuid: "3487895bdebd4925b1e0ee7b961c18d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90873f7a558149f094dd37e015a2d8f8"
I20260807 18:16:48.467728 9874 raft_consensus.cc:3037] T 97473341374741d6b3a437769d5d74a2 P b3f50a9f91964b79b66d73c38ac9ab47 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:16:48.467876 10248 raft_consensus.cc:3037] T 97473341374741d6b3a437769d5d74a2 P 90873f7a558149f094dd37e015a2d8f8 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:16:48.470161 9874 raft_consensus.cc:2454] T 97473341374741d6b3a437769d5d74a2 P b3f50a9f91964b79b66d73c38ac9ab47 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3487895bdebd4925b1e0ee7b961c18d9 in term 1.
I20260807 18:16:48.470337 10248 raft_consensus.cc:2454] T 97473341374741d6b3a437769d5d74a2 P 90873f7a558149f094dd37e015a2d8f8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3487895bdebd4925b1e0ee7b961c18d9 in term 1.
I20260807 18:16:48.470610 9974 leader_election.cc:302] T 97473341374741d6b3a437769d5d74a2 P 3487895bdebd4925b1e0ee7b961c18d9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3487895bdebd4925b1e0ee7b961c18d9, b3f50a9f91964b79b66d73c38ac9ab47; no voters:
I20260807 18:16:48.470940 10342 raft_consensus.cc:2781] T 97473341374741d6b3a437769d5d74a2 P 3487895bdebd4925b1e0ee7b961c18d9 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:16:48.471426 10342 raft_consensus.cc:686] T 97473341374741d6b3a437769d5d74a2 P 3487895bdebd4925b1e0ee7b961c18d9 [term 1 LEADER]: Becoming Leader. State: Replica: 3487895bdebd4925b1e0ee7b961c18d9, State: Running, Role: LEADER
I20260807 18:16:48.471903 10342 consensus_queue.cc:237] T 97473341374741d6b3a437769d5d74a2 P 3487895bdebd4925b1e0ee7b961c18d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3f50a9f91964b79b66d73c38ac9ab47" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 34909 } } peers { permanent_uuid: "90873f7a558149f094dd37e015a2d8f8" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 33869 } } peers { permanent_uuid: "3487895bdebd4925b1e0ee7b961c18d9" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 42929 } }
I20260807 18:16:48.477036 9677 catalog_manager.cc:5495] T 97473341374741d6b3a437769d5d74a2 P 3487895bdebd4925b1e0ee7b961c18d9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3487895bdebd4925b1e0ee7b961c18d9 (127.18.154.2). New cstate: current_term: 1 leader_uuid: "3487895bdebd4925b1e0ee7b961c18d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3f50a9f91964b79b66d73c38ac9ab47" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90873f7a558149f094dd37e015a2d8f8" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 33869 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3487895bdebd4925b1e0ee7b961c18d9" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 42929 } health_report { overall_health: HEALTHY } } }
I20260807 18:16:48.512256 10330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:48.506734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43526 (local address 127.18.154.2:42929)
0807 18:16:48.506925 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0807 18:16:48.506929 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:16:48.506946 (+ 17us) server_negotiation.cc:407] Connection header received
0807 18:16:48.507035 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:16:48.507039 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:16:48.507126 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:16:48.507258 (+ 132us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:16:48.507702 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:16:48.508448 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:16:48.510142 (+ 1694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:16:48.510540 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:16:48.510635 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:16:48.511120 (+ 485us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:16:48.511693 (+ 573us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:16:48.511808 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0807 18:16:48.511953 (+ 145us) server_negotiation.cc:299] Negotiation successful
0807 18:16:48.512025 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260807 18:16:48.521278 10248 raft_consensus.cc:1261] T 97473341374741d6b3a437769d5d74a2 P 90873f7a558149f094dd37e015a2d8f8 [term 1 FOLLOWER]: Refusing update from remote peer 3487895bdebd4925b1e0ee7b961c18d9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:16:48.521275 9874 raft_consensus.cc:1261] T 97473341374741d6b3a437769d5d74a2 P b3f50a9f91964b79b66d73c38ac9ab47 [term 1 FOLLOWER]: Refusing update from remote peer 3487895bdebd4925b1e0ee7b961c18d9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:16:48.522431 10347 consensus_queue.cc:1035] T 97473341374741d6b3a437769d5d74a2 P 3487895bdebd4925b1e0ee7b961c18d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3f50a9f91964b79b66d73c38ac9ab47" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:16:48.522800 10342 consensus_queue.cc:1035] T 97473341374741d6b3a437769d5d74a2 P 3487895bdebd4925b1e0ee7b961c18d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90873f7a558149f094dd37e015a2d8f8" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 33869 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:16:48.533418 10349 mvcc.cc:205] Tried to move back new op lower bound from 7315974588491272192 to 7315974588304969728. Current Snapshot: MvccSnapshot[applied={T|T < 7315974588491272192}]
I20260807 18:16:48.535149 10351 mvcc.cc:205] Tried to move back new op lower bound from 7315974588491272192 to 7315974588304969728. Current Snapshot: MvccSnapshot[applied={T|T < 7315974588491272192}]
I20260807 18:16:48.550544 9677 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56792:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:16:48.550838 9677 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56792:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:16:48.554322 9677 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 74b7e49ff11046dba46d21f00d9fd15a: Sending DeleteTablet for 3 replicas of tablet 97473341374741d6b3a437769d5d74a2
I20260807 18:16:48.555704 10035 tablet_service.cc:1514] Processing DeleteTablet for tablet 97473341374741d6b3a437769d5d74a2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:16:48 UTC) from {username='kudu', principal='kudu/127.18.154.62@KRBTEST.COM'} at 127.0.0.1:43520
I20260807 18:16:48.555750 9847 tablet_service.cc:1514] Processing DeleteTablet for tablet 97473341374741d6b3a437769d5d74a2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:16:48 UTC) from {username='kudu', principal='kudu/127.18.154.62@KRBTEST.COM'} at 127.0.0.1:34626
I20260807 18:16:48.555807 10221 tablet_service.cc:1514] Processing DeleteTablet for tablet 97473341374741d6b3a437769d5d74a2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:16:48 UTC) from {username='kudu', principal='kudu/127.18.154.62@KRBTEST.COM'} at 127.0.0.1:32978
I20260807 18:16:48.556645 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 9737
I20260807 18:16:48.571516 10367 tablet_replica.cc:330] T 97473341374741d6b3a437769d5d74a2 P b3f50a9f91964b79b66d73c38ac9ab47: stopping tablet replica
I20260807 18:16:48.571552 10364 tablet_replica.cc:330] T 97473341374741d6b3a437769d5d74a2 P 3487895bdebd4925b1e0ee7b961c18d9: stopping tablet replica
I20260807 18:16:48.572110 10364 raft_consensus.cc:2229] T 97473341374741d6b3a437769d5d74a2 P 3487895bdebd4925b1e0ee7b961c18d9 [term 1 LEADER]: Raft consensus shutting down.
I20260807 18:16:48.572212 10367 raft_consensus.cc:2229] T 97473341374741d6b3a437769d5d74a2 P b3f50a9f91964b79b66d73c38ac9ab47 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:16:48.572930 10367 raft_consensus.cc:2258] T 97473341374741d6b3a437769d5d74a2 P b3f50a9f91964b79b66d73c38ac9ab47 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:16:48.572913 10364 raft_consensus.cc:2258] T 97473341374741d6b3a437769d5d74a2 P 3487895bdebd4925b1e0ee7b961c18d9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:16:48.573825 10366 tablet_replica.cc:330] T 97473341374741d6b3a437769d5d74a2 P 90873f7a558149f094dd37e015a2d8f8: stopping tablet replica
I20260807 18:16:48.574297 10366 raft_consensus.cc:2229] T 97473341374741d6b3a437769d5d74a2 P 90873f7a558149f094dd37e015a2d8f8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:16:48.574863 10366 raft_consensus.cc:2258] T 97473341374741d6b3a437769d5d74a2 P 90873f7a558149f094dd37e015a2d8f8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:16:48.577059 10364 ts_tablet_manager.cc:1905] T 97473341374741d6b3a437769d5d74a2 P 3487895bdebd4925b1e0ee7b961c18d9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:16:48.577157 10366 ts_tablet_manager.cc:1905] T 97473341374741d6b3a437769d5d74a2 P 90873f7a558149f094dd37e015a2d8f8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:16:48.583040 10328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:48.557695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34642 (local address 127.18.154.1:34909)
0807 18:16:48.558440 (+ 745us) server_negotiation.cc:206] Beginning negotiation
0807 18:16:48.558449 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:16:48.558472 (+ 23us) server_negotiation.cc:407] Connection header received
0807 18:16:48.558545 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:16:48.558552 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:16:48.558683 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:16:48.558810 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:16:48.561792 (+ 2982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:16:48.563380 (+ 1588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:16:48.566041 (+ 2661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:16:48.567017 (+ 976us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:16:48.567171 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:16:48.574203 (+ 7032us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:16:48.574217 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:16:48.574235 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:16:48.574282 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:16:48.578602 (+ 4320us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:16:48.579535 (+ 933us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:16:48.579547 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:16:48.579552 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:16:48.579753 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:16:48.580210 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:16:48.580215 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:16:48.580219 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:16:48.580661 (+ 442us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:16:48.581044 (+ 383us) server_negotiation.cc:1035] Waiting for connection context
0807 18:16:48.582395 (+ 1351us) server_negotiation.cc:299] Negotiation successful
0807 18:16:48.582623 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":557}
I20260807 18:16:48.584031 9929 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:16:48.584725 10366 ts_tablet_manager.cc:1918] T 97473341374741d6b3a437769d5d74a2 P 90873f7a558149f094dd37e015a2d8f8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:16:48.585601 10364 ts_tablet_manager.cc:1918] T 97473341374741d6b3a437769d5d74a2 P 3487895bdebd4925b1e0ee7b961c18d9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:16:48.585810 10366 log.cc:1196] T 97473341374741d6b3a437769d5d74a2 P 90873f7a558149f094dd37e015a2d8f8: Deleting WAL directory at /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/ts-2/wal/wals/97473341374741d6b3a437769d5d74a2
I20260807 18:16:48.585892 10364 log.cc:1196] T 97473341374741d6b3a437769d5d74a2 P 3487895bdebd4925b1e0ee7b961c18d9: Deleting WAL directory at /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/ts-1/wal/wals/97473341374741d6b3a437769d5d74a2
I20260807 18:16:48.586622 10364 ts_tablet_manager.cc:1939] T 97473341374741d6b3a437769d5d74a2 P 3487895bdebd4925b1e0ee7b961c18d9: Deleting consensus metadata
I20260807 18:16:48.586686 10366 ts_tablet_manager.cc:1939] T 97473341374741d6b3a437769d5d74a2 P 90873f7a558149f094dd37e015a2d8f8: Deleting consensus metadata
I20260807 18:16:48.588817 9663 catalog_manager.cc:4841] TS 3487895bdebd4925b1e0ee7b961c18d9 (127.18.154.2:42929): tablet 97473341374741d6b3a437769d5d74a2 (table test-table [id=2365d5179ea247bb8ddb601ff096752b]) successfully deleted
I20260807 18:16:48.589253 9663 catalog_manager.cc:4841] TS 90873f7a558149f094dd37e015a2d8f8 (127.18.154.3:33869): tablet 97473341374741d6b3a437769d5d74a2 (table test-table [id=2365d5179ea247bb8ddb601ff096752b]) successfully deleted
I20260807 18:16:48.585604 10367 ts_tablet_manager.cc:1905] T 97473341374741d6b3a437769d5d74a2 P b3f50a9f91964b79b66d73c38ac9ab47: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:16:48.682279 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 9737
W20260807 18:16:48.712637 9664 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.154.1:34909 (error 108)
I20260807 18:16:48.712947 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 9942
I20260807 18:16:48.715030 10331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:48.714514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.1:34909 (local address 127.0.0.1:34656)
0807 18:16:48.714729 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0807 18:16:48.714848 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.1:34909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
W20260807 18:16:48.715476 9664 catalog_manager.cc:4584] TS b3f50a9f91964b79b66d73c38ac9ab47 (127.18.154.1:34909): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 97473341374741d6b3a437769d5d74a2: Network error: Client connection negotiation failed: client connection to 127.18.154.1:34909: connect: Connection refused (error 111)
I20260807 18:16:48.733068 10330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:48.713940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43532 (local address 127.18.154.2:42929)
0807 18:16:48.714129 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0807 18:16:48.714133 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:16:48.714158 (+ 25us) server_negotiation.cc:407] Connection header received
0807 18:16:48.714215 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:16:48.714218 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:16:48.714282 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:16:48.714435 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:16:48.716326 (+ 1891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:16:48.717317 (+ 991us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:16:48.718917 (+ 1600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:16:48.719421 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:16:48.719573 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:16:48.726594 (+ 7021us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:16:48.726607 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:16:48.726624 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:16:48.726668 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:16:48.729972 (+ 3304us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:16:48.730736 (+ 764us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:16:48.730745 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:16:48.730749 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:16:48.730907 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:16:48.731392 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:16:48.731400 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:16:48.731410 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:16:48.731810 (+ 400us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:16:48.732178 (+ 368us) server_negotiation.cc:1035] Waiting for connection context
0807 18:16:48.732421 (+ 243us) server_negotiation.cc:299] Negotiation successful
0807 18:16:48.732658 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260807 18:16:48.733937 10119 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:16:48.769516 10331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:48.768927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.1:34909 (local address 127.0.0.1:34660)
0807 18:16:48.769232 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0807 18:16:48.769352 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.1:34909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101}
I20260807 18:16:48.770812 10331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:48.770449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.1:34909 (local address 127.0.0.1:34664)
0807 18:16:48.770594 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0807 18:16:48.770677 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.1:34909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260807 18:16:48.831604 10331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:48.831035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.1:34909 (local address 127.0.0.1:34666)
0807 18:16:48.831318 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0807 18:16:48.831425 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.1:34909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112}
I20260807 18:16:48.832895 10331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:48.832468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.1:34909 (local address 127.0.0.1:34670)
0807 18:16:48.832657 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0807 18:16:48.832749 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.1:34909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260807 18:16:48.850011 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 9942
I20260807 18:16:48.874112 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 10131
I20260807 18:16:48.888391 10329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:48.874924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32994 (local address 127.18.154.3:33869)
0807 18:16:48.875116 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0807 18:16:48.875124 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:16:48.875250 (+ 126us) server_negotiation.cc:407] Connection header received
0807 18:16:48.875496 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:16:48.875500 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:16:48.875592 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:16:48.875716 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:16:48.877108 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:16:48.878294 (+ 1186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:16:48.879742 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:16:48.880127 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:16:48.880223 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:16:48.883069 (+ 2846us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:16:48.883084 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:16:48.883101 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:16:48.883137 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:16:48.885915 (+ 2778us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:16:48.886558 (+ 643us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:16:48.886565 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:16:48.886567 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:16:48.886713 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:16:48.887029 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:16:48.887032 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:16:48.887034 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:16:48.887399 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:16:48.887639 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0807 18:16:48.887867 (+ 228us) server_negotiation.cc:299] Negotiation successful
0807 18:16:48.888026 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260807 18:16:48.889061 10306 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:16:48.914307 10331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:48.913779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.1:34909 (local address 127.0.0.1:34680)
0807 18:16:48.913995 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0807 18:16:48.914089 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.1:34909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260807 18:16:48.915378 10331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:48.914996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.1:34909 (local address 127.0.0.1:34684)
0807 18:16:48.915194 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0807 18:16:48.915273 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.1:34909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97}
I20260807 18:16:48.981231 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 10131
I20260807 18:16:49.002413 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 9637
I20260807 18:16:49.003639 9727 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:16:49.121954 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 9637
I20260807 18:16:49.122044 10331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:16:49.121120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.1:34909 (local address 127.0.0.1:34696)
0807 18:16:49.121344 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0807 18:16:49.121477 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.1:34909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260807 18:17:09.152159 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.154.62:46089
--webserver_interface=127.18.154.62
--webserver_port=44395
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.62.keytab
--principal=kudu/127.18.154.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.154.20:44143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.154.62:46089
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:17:09.512610 10377 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:17:09.512974 10377 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:17:09.513145 10377 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:17:09.520462 10377 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260807 18:17:09.520614 10377 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260807 18:17:09.520689 10377 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260807 18:17:09.520777 10377 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:17:09.520874 10377 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:17:09.520998 10377 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:17:09.521088 10377 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:17:09.521198 10377 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:17:09.531575 10377 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.154.20:44143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.18.154.62:46089
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.62.keytab
--principal=kudu/127.18.154.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.154.62:46089
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.154.62
--webserver_port=44395
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:17:09.533038 10377 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:17:09.534502 10377 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:17:09.547739 10386 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:17:09.547772 10388 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:17:09.547863 10390 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:17:09.548542 10377 server_base.cc:1034] running on GCE node
I20260807 18:17:09.549407 10377 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:17:09.550981 10377 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:17:09.552210 10377 hybrid_clock.cc:648] HybridClock initialized: now 1786126629552160 us; error 49 us; skew 500 ppm
I20260807 18:17:09.552613 10377 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:17:09 dist-test-slave-gkw7 krb5kdc[8868](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126629, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:17:09.556012 10377 init.cc:377] Logged in from keytab as kudu/127.18.154.62@KRBTEST.COM (short username kudu)
I20260807 18:17:09.557801 10377 webserver.cc:460] Webserver started at http://127.18.154.62:44395/ using document root <none> and password file <none>
I20260807 18:17:09.558595 10377 fs_manager.cc:359] Metadata directory not provided
I20260807 18:17:09.558741 10377 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:17:09.564980 10377 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 18:17:09.568200 10400 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:17:09.569644 10377 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:17:09.569864 10377 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/master-0/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/master-0/wal
uuid: "74b7e49ff11046dba46d21f00d9fd15a"
format_stamp: "Formatted at 2026-08-07 18:16:23 on dist-test-slave-gkw7"
server_key: "25521df0306039dcaea0a5175ac320c4"
server_key_iv: "3c597486b4bad2870224c9e0ab5dee35"
server_key_version: "encryptionkey@0"
I20260807 18:17:09.570935 10377 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:17:09.605351 10377 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:17:09.610019 10377 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:17:09.610347 10377 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:17:09.631055 10377 rpc_server.cc:307] RPC server started. Bound to: 127.18.154.62:46089
I20260807 18:17:09.631093 10473 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.154.62:46089 every 8 connection(s)
I20260807 18:17:09.632717 10377 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
I20260807 18:17:09.633635 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 10377
I20260807 18:17:09.634709 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.154.1:34909
--local_ip_for_outbound_sockets=127.18.154.1
--tserver_master_addrs=127.18.154.62:46089
--webserver_port=40111
--webserver_interface=127.18.154.1
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.1.keytab
--principal=kudu/127.18.154.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.154.20:44143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 18:17:09.640801 10474 sys_catalog.cc:263] Verifying existing consensus state
I20260807 18:17:09.644461 10474 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 74b7e49ff11046dba46d21f00d9fd15a: Bootstrap starting.
I20260807 18:17:09.675611 10474 log.cc:824] T 00000000000000000000000000000000 P 74b7e49ff11046dba46d21f00d9fd15a: Log is configured to *not* fsync() on all Append() calls
I20260807 18:17:09.697557 10474 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 74b7e49ff11046dba46d21f00d9fd15a: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 18:17:09.698318 10474 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 74b7e49ff11046dba46d21f00d9fd15a: Bootstrap complete.
I20260807 18:17:09.713326 10474 raft_consensus.cc:348] T 00000000000000000000000000000000 P 74b7e49ff11046dba46d21f00d9fd15a [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74b7e49ff11046dba46d21f00d9fd15a" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 46089 } }
I20260807 18:17:09.715045 10474 raft_consensus.cc:729] T 00000000000000000000000000000000 P 74b7e49ff11046dba46d21f00d9fd15a [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74b7e49ff11046dba46d21f00d9fd15a, State: Initialized, Role: FOLLOWER
I20260807 18:17:09.716017 10474 consensus_queue.cc:260] T 00000000000000000000000000000000 P 74b7e49ff11046dba46d21f00d9fd15a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74b7e49ff11046dba46d21f00d9fd15a" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 46089 } }
I20260807 18:17:09.716375 10474 raft_consensus.cc:388] T 00000000000000000000000000000000 P 74b7e49ff11046dba46d21f00d9fd15a [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:17:09.716516 10474 raft_consensus.cc:482] T 00000000000000000000000000000000 P 74b7e49ff11046dba46d21f00d9fd15a [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:17:09.716719 10474 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 74b7e49ff11046dba46d21f00d9fd15a [term 2 FOLLOWER]: Advancing to term 3
I20260807 18:17:09.722729 10474 raft_consensus.cc:504] T 00000000000000000000000000000000 P 74b7e49ff11046dba46d21f00d9fd15a [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74b7e49ff11046dba46d21f00d9fd15a" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 46089 } }
I20260807 18:17:09.723420 10474 leader_election.cc:302] T 00000000000000000000000000000000 P 74b7e49ff11046dba46d21f00d9fd15a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 74b7e49ff11046dba46d21f00d9fd15a; no voters:
I20260807 18:17:09.724167 10474 leader_election.cc:288] T 00000000000000000000000000000000 P 74b7e49ff11046dba46d21f00d9fd15a [CANDIDATE]: Term 3 election: Requested vote from peers
I20260807 18:17:09.724361 10480 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 74b7e49ff11046dba46d21f00d9fd15a [term 3 FOLLOWER]: Leader election won for term 3
I20260807 18:17:09.725940 10480 raft_consensus.cc:686] T 00000000000000000000000000000000 P 74b7e49ff11046dba46d21f00d9fd15a [term 3 LEADER]: Becoming Leader. State: Replica: 74b7e49ff11046dba46d21f00d9fd15a, State: Running, Role: LEADER
I20260807 18:17:09.726537 10480 consensus_queue.cc:237] T 00000000000000000000000000000000 P 74b7e49ff11046dba46d21f00d9fd15a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74b7e49ff11046dba46d21f00d9fd15a" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 46089 } }
I20260807 18:17:09.726637 10474 sys_catalog.cc:564] T 00000000000000000000000000000000 P 74b7e49ff11046dba46d21f00d9fd15a [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:17:09.734925 10481 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74b7e49ff11046dba46d21f00d9fd15a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "74b7e49ff11046dba46d21f00d9fd15a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74b7e49ff11046dba46d21f00d9fd15a" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 46089 } } }
I20260807 18:17:09.734843 10483 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74b7e49ff11046dba46d21f00d9fd15a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 74b7e49ff11046dba46d21f00d9fd15a. Latest consensus state: current_term: 3 leader_uuid: "74b7e49ff11046dba46d21f00d9fd15a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74b7e49ff11046dba46d21f00d9fd15a" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 46089 } } }
I20260807 18:17:09.735303 10483 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74b7e49ff11046dba46d21f00d9fd15a [sys.catalog]: This master's current role is: LEADER
I20260807 18:17:09.735304 10481 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74b7e49ff11046dba46d21f00d9fd15a [sys.catalog]: This master's current role is: LEADER
I20260807 18:17:09.742753 10488 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:17:09.751787 10488 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:17:09.753413 10488 catalog_manager.cc:1259] Loaded cluster ID: 64638edcd2e6457cb96fce940f3801a9
I20260807 18:17:09.753628 10488 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:17:09.757932 10488 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:17:09.759644 10488 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 74b7e49ff11046dba46d21f00d9fd15a: Loaded TSK: 0
I20260807 18:17:09.759871 10488 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 74b7e49ff11046dba46d21f00d9fd15a: Loaded TSK: 1
F20260807 18:17:09.761283 10488 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7eff149da9ad google::LogMessage::Fail() at ??:0
@ 0x7eff149dd88a google::LogMessage::SendToLog() at ??:0
@ 0x7eff149da4de google::LogMessage::Flush() at ??:0
@ 0x7eff149dba19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7eff1873e7d2 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7eff1872d73e kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7eff2a4091c0 kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7eff2a408783 kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7eff2a4942a7 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_10::operator()() at ??:0
I20260807 18:17:09.797057 10506 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
@ 0x7eff2a4940c4 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x7eff2a4de5e9 std::function<>::operator()() at ??:0
@ 0x7eff2a40ccdd kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7eff2a40ac36 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7eff2a492610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7eff2a492439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7eff17626345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7eff1762991c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7eff17629799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7eff17604eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7eff1a8de6db start_thread at ??:0
@ 0x7eff1387461f clone at ??:0
W20260807 18:17:10.093324 10476 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:17:10.093693 10476 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:17:10.093887 10476 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:17:10.101608 10476 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:17:10.101750 10476 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:17:10.101980 10476 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.154.1
I20260807 18:17:10.112331 10476 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.154.20:44143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.1.keytab
--principal=kudu/127.18.154.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.154.1:34909
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.154.1
--webserver_port=40111
--tserver_master_addrs=127.18.154.62:46089
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.154.1
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:17:10.113746 10476 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:17:10.115115 10476 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:17:10.128921 10547 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:17:10.129014 10545 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:17:10.128979 10543 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:17:10.130174 10476 server_base.cc:1034] running on GCE node
I20260807 18:17:10.130838 10476 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:17:10.131850 10476 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:17:10.133080 10476 hybrid_clock.cc:648] HybridClock initialized: now 1786126630133035 us; error 60 us; skew 500 ppm
I20260807 18:17:10.133421 10476 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:17:10 dist-test-slave-gkw7 krb5kdc[8868](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126630, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:17:10.136709 10476 init.cc:377] Logged in from keytab as kudu/127.18.154.1@KRBTEST.COM (short username kudu)
I20260807 18:17:10.138311 10476 webserver.cc:460] Webserver started at http://127.18.154.1:40111/ using document root <none> and password file <none>
I20260807 18:17:10.139036 10476 fs_manager.cc:359] Metadata directory not provided
I20260807 18:17:10.139209 10476 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:17:10.144724 10476 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 18:17:10.148017 10559 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:17:10.149518 10476 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 18:17:10.149739 10476 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/ts-0/wal
uuid: "b3f50a9f91964b79b66d73c38ac9ab47"
format_stamp: "Formatted at 2026-08-07 18:16:24 on dist-test-slave-gkw7"
server_key: "9bcaaa5a6d0b685de9523261f10c7145"
server_key_iv: "6e711d227088d40fdf26e493e31bed16"
server_key_version: "encryptionkey@0"
I20260807 18:17:10.150697 10476 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:17:10.173095 10476 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:17:10.177894 10476 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:17:10.178133 10476 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:17:10.179139 10476 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:17:10.181032 10570 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260807 18:17:10.187218 10476 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260807 18:17:10.187369 10476 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.007s user 0.001s sys 0.000s
I20260807 18:17:10.187533 10476 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260807 18:17:10.190963 10476 ts_tablet_manager.cc:610] Registered 1 tablets
I20260807 18:17:10.191087 10476 ts_tablet_manager.cc:589] Time spent register tablets: real 0.004s user 0.002s sys 0.000s
I20260807 18:17:10.191444 10570 tablet_bootstrap.cc:492] T 97473341374741d6b3a437769d5d74a2 P b3f50a9f91964b79b66d73c38ac9ab47: Bootstrap starting.
I20260807 18:17:10.200439 10570 log.cc:824] T 97473341374741d6b3a437769d5d74a2 P b3f50a9f91964b79b66d73c38ac9ab47: Log is configured to *not* fsync() on all Append() calls
I20260807 18:17:10.216631 10570 tablet_bootstrap.cc:492] T 97473341374741d6b3a437769d5d74a2 P b3f50a9f91964b79b66d73c38ac9ab47: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 18:17:10.217517 10570 tablet_bootstrap.cc:492] T 97473341374741d6b3a437769d5d74a2 P b3f50a9f91964b79b66d73c38ac9ab47: Bootstrap complete.
I20260807 18:17:10.218205 10570 ts_tablet_manager.cc:1397] T 97473341374741d6b3a437769d5d74a2 P b3f50a9f91964b79b66d73c38ac9ab47: Time spent bootstrapping tablet: real 0.027s user 0.025s sys 0.000s
I20260807 18:17:10.227643 10570 raft_consensus.cc:348] T 97473341374741d6b3a437769d5d74a2 P b3f50a9f91964b79b66d73c38ac9ab47 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3f50a9f91964b79b66d73c38ac9ab47" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 34909 } } peers { permanent_uuid: "90873f7a558149f094dd37e015a2d8f8" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 33869 } } peers { permanent_uuid: "3487895bdebd4925b1e0ee7b961c18d9" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 42929 } }
I20260807 18:17:10.228926 10570 raft_consensus.cc:729] T 97473341374741d6b3a437769d5d74a2 P b3f50a9f91964b79b66d73c38ac9ab47 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3f50a9f91964b79b66d73c38ac9ab47, State: Initialized, Role: FOLLOWER
I20260807 18:17:10.229769 10570 consensus_queue.cc:260] T 97473341374741d6b3a437769d5d74a2 P b3f50a9f91964b79b66d73c38ac9ab47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3f50a9f91964b79b66d73c38ac9ab47" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 34909 } } peers { permanent_uuid: "90873f7a558149f094dd37e015a2d8f8" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 33869 } } peers { permanent_uuid: "3487895bdebd4925b1e0ee7b961c18d9" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 42929 } }
I20260807 18:17:10.231976 10570 ts_tablet_manager.cc:1428] T 97473341374741d6b3a437769d5d74a2 P b3f50a9f91964b79b66d73c38ac9ab47: Time spent starting tablet: real 0.014s user 0.013s sys 0.000s
I20260807 18:17:10.239641 10476 rpc_server.cc:307] RPC server started. Bound to: 127.18.154.1:34909
I20260807 18:17:10.239692 10718 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.154.1:34909 every 8 connection(s)
I20260807 18:17:10.241003 10476 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/ts-0/data/info.pb
I20260807 18:17:10.246001 10721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:10.243962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:46089 (local address 127.18.154.1:54365)
0807 18:17:10.244752 (+ 790us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:10.244988 (+ 236us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.62:46089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":524,"thread_start_us":316,"threads_started":1}
W20260807 18:17:10.246747 10719 heartbeater.cc:644] Failed to heartbeat to 127.18.154.62:46089 (0 consecutive failures): Network error: Failed to ping master at 127.18.154.62:46089: Client connection negotiation failed: client connection to 127.18.154.62:46089: connect: Connection refused (error 111)
I20260807 18:17:10.247574 10721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:10.247246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:46089 (local address 127.18.154.1:34747)
0807 18:17:10.247402 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:10.247475 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.62:46089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260807 18:17:10.248409 10721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:10.248072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:46089 (local address 127.18.154.1:45077)
0807 18:17:10.248217 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:10.248297 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.62:46089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260807 18:17:10.248649 10719 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260807 18:17:10.249770 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 10476
I20260807 18:17:10.250442 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.154.2:42929
--local_ip_for_outbound_sockets=127.18.154.2
--tserver_master_addrs=127.18.154.62:46089
--webserver_port=44051
--webserver_interface=127.18.154.2
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.2.keytab
--principal=kudu/127.18.154.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.154.20:44143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:17:10.612331 10722 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:17:10.612643 10722 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:17:10.612738 10722 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:17:10.620029 10722 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:17:10.620151 10722 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:17:10.620266 10722 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.154.2
I20260807 18:17:10.630172 10722 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.154.20:44143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.2.keytab
--principal=kudu/127.18.154.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.154.2:42929
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.154.2
--webserver_port=44051
--tserver_master_addrs=127.18.154.62:46089
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.154.2
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:17:10.631438 10722 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:17:10.632784 10722 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:17:10.646860 10730 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:17:10.646942 10729 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:17:10.646912 10733 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:17:10.648469 10722 server_base.cc:1034] running on GCE node
I20260807 18:17:10.649199 10722 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:17:10.650604 10722 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:17:10.651890 10722 hybrid_clock.cc:648] HybridClock initialized: now 1786126630651839 us; error 53 us; skew 500 ppm
I20260807 18:17:10.652274 10722 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:17:10 dist-test-slave-gkw7 krb5kdc[8868](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126630, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:17:10.656016 10722 init.cc:377] Logged in from keytab as kudu/127.18.154.2@KRBTEST.COM (short username kudu)
I20260807 18:17:10.657765 10722 webserver.cc:460] Webserver started at http://127.18.154.2:44051/ using document root <none> and password file <none>
I20260807 18:17:10.658505 10722 fs_manager.cc:359] Metadata directory not provided
I20260807 18:17:10.658630 10722 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:17:10.664673 10722 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.003s
I20260807 18:17:10.667656 10743 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:17:10.669088 10722 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 18:17:10.669337 10722 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/ts-1/wal
uuid: "3487895bdebd4925b1e0ee7b961c18d9"
format_stamp: "Formatted at 2026-08-07 18:16:24 on dist-test-slave-gkw7"
server_key: "5479e736d511ce1d944ebffc9f018b16"
server_key_iv: "3268433d2eac4524a60aef8bd486c087"
server_key_version: "encryptionkey@0"
I20260807 18:17:10.670449 10722 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:17:10.691519 10722 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:17:10.696170 10722 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:17:10.696460 10722 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:17:10.697644 10722 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:17:10.699067 10722 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:17:10.699194 10722 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:17:10.699344 10722 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:17:10.699440 10722 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:17:10.734203 10722 rpc_server.cc:307] RPC server started. Bound to: 127.18.154.2:42929
I20260807 18:17:10.734263 10894 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.154.2:42929 every 8 connection(s)
I20260807 18:17:10.735725 10722 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/ts-1/data/info.pb
I20260807 18:17:10.741076 10898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:10.738838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:46089 (local address 127.18.154.2:54559)
0807 18:17:10.739650 (+ 812us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:10.740025 (+ 375us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.62:46089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":466,"thread_start_us":286,"threads_started":1}
W20260807 18:17:10.741838 10895 heartbeater.cc:644] Failed to heartbeat to 127.18.154.62:46089 (0 consecutive failures): Network error: Failed to ping master at 127.18.154.62:46089: Client connection negotiation failed: client connection to 127.18.154.62:46089: connect: Connection refused (error 111)
I20260807 18:17:10.742439 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 10722
I20260807 18:17:10.742947 10898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:10.742479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:46089 (local address 127.18.154.2:44637)
0807 18:17:10.742700 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:10.742785 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.62:46089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104}
I20260807 18:17:10.743280 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.154.3:33869
--local_ip_for_outbound_sockets=127.18.154.3
--tserver_master_addrs=127.18.154.62:46089
--webserver_port=43187
--webserver_interface=127.18.154.3
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.3.keytab
--principal=kudu/127.18.154.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.154.20:44143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 18:17:10.744043 10898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:10.743615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:46089 (local address 127.18.154.2:55523)
0807 18:17:10.743783 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:10.743884 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.62:46089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260807 18:17:10.744374 10895 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 18:17:11.094177 10899 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:17:11.094518 10899 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:17:11.094694 10899 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:17:11.102211 10899 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:17:11.102380 10899 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:17:11.102578 10899 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.154.3
I20260807 18:17:11.112669 10899 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.154.20:44143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.3.keytab
--principal=kudu/127.18.154.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.154.3:33869
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.154.3
--webserver_port=43187
--tserver_master_addrs=127.18.154.62:46089
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.154.3
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:17:11.114182 10899 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:17:11.115545 10899 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:17:11.129608 10911 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:17:11.129673 10907 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:17:11.129688 10913 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:17:11.130533 10899 server_base.cc:1034] running on GCE node
I20260807 18:17:11.131165 10899 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:17:11.132172 10899 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:17:11.133404 10899 hybrid_clock.cc:648] HybridClock initialized: now 1786126631133340 us; error 70 us; skew 500 ppm
I20260807 18:17:11.133787 10899 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:17:11 dist-test-slave-gkw7 krb5kdc[8868](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126631, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:17:11.137028 10899 init.cc:377] Logged in from keytab as kudu/127.18.154.3@KRBTEST.COM (short username kudu)
I20260807 18:17:11.138556 10899 webserver.cc:460] Webserver started at http://127.18.154.3:43187/ using document root <none> and password file <none>
I20260807 18:17:11.139233 10899 fs_manager.cc:359] Metadata directory not provided
I20260807 18:17:11.139343 10899 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:17:11.144969 10899 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 18:17:11.147948 10923 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:17:11.149430 10899 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 18:17:11.149659 10899 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/ts-2/wal
uuid: "90873f7a558149f094dd37e015a2d8f8"
format_stamp: "Formatted at 2026-08-07 18:16:25 on dist-test-slave-gkw7"
server_key: "11c2a1e3e5528eb5d85d9fe5a4b412a4"
server_key_iv: "9781bbb23881a1c542e42a6b05af0077"
server_key_version: "encryptionkey@0"
I20260807 18:17:11.150620 10899 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:17:11.174366 10899 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:17:11.178746 10899 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:17:11.178995 10899 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:17:11.179970 10899 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:17:11.181341 10899 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:17:11.181439 10899 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:17:11.181516 10899 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:17:11.181648 10899 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
I20260807 18:17:11.217016 10899 rpc_server.cc:307] RPC server started. Bound to: 127.18.154.3:33869
I20260807 18:17:11.217123 11071 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.154.3:33869 every 8 connection(s)
I20260807 18:17:11.218591 10899 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786126328808105-19048-0/minicluster-data/ts-2/data/info.pb
I20260807 18:17:11.223642 11074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:11.221784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:46089 (local address 127.18.154.3:39107)
0807 18:17:11.222548 (+ 764us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:11.222800 (+ 252us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.62:46089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":486,"thread_start_us":279,"threads_started":1}
W20260807 18:17:11.224428 11072 heartbeater.cc:644] Failed to heartbeat to 127.18.154.62:46089 (0 consecutive failures): Network error: Failed to ping master at 127.18.154.62:46089: Client connection negotiation failed: client connection to 127.18.154.62:46089: connect: Connection refused (error 111)
I20260807 18:17:11.225098 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 10899
I20260807 18:17:11.225351 11074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:11.224931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:46089 (local address 127.18.154.3:36083)
0807 18:17:11.225117 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:11.225209 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.62:46089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260807 18:17:11.226297 11074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:11.225927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:46089 (local address 127.18.154.3:59735)
0807 18:17:11.226069 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:11.226149 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.62:46089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260807 18:17:11.226552 11072 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260807 18:17:11.250108 11076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:11.249236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:46089 (local address 127.18.154.1:33955)
0807 18:17:11.249815 (+ 579us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:11.249891 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.62:46089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":315,"threads_started":1}
I20260807 18:17:11.745985 11079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:11.744986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:46089 (local address 127.18.154.2:40105)
0807 18:17:11.745626 (+ 640us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:11.745743 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.62:46089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":474,"thread_start_us":338,"threads_started":1}
I20260807 18:17:11.860796 11080 raft_consensus.cc:482] T 97473341374741d6b3a437769d5d74a2 P b3f50a9f91964b79b66d73c38ac9ab47 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:17:11.861140 11080 raft_consensus.cc:504] T 97473341374741d6b3a437769d5d74a2 P b3f50a9f91964b79b66d73c38ac9ab47 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3f50a9f91964b79b66d73c38ac9ab47" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 34909 } } peers { permanent_uuid: "90873f7a558149f094dd37e015a2d8f8" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 33869 } } peers { permanent_uuid: "3487895bdebd4925b1e0ee7b961c18d9" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 42929 } }
I20260807 18:17:11.862887 11080 leader_election.cc:288] T 97473341374741d6b3a437769d5d74a2 P b3f50a9f91964b79b66d73c38ac9ab47 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 90873f7a558149f094dd37e015a2d8f8 (127.18.154.3:33869), 3487895bdebd4925b1e0ee7b961c18d9 (127.18.154.2:42929)
Aug 07 18:17:11 dist-test-slave-gkw7 krb5kdc[8868](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126630, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.1@KRBTEST.COM for kudu/127.18.154.3@KRBTEST.COM
Aug 07 18:17:11 dist-test-slave-gkw7 krb5kdc[8868](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126630, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.1@KRBTEST.COM for kudu/127.18.154.2@KRBTEST.COM
I20260807 18:17:11.886191 11082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:11.863411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.1:47577 (local address 127.18.154.3:33869)
0807 18:17:11.863952 (+ 541us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:11.863965 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:11.864012 (+ 47us) server_negotiation.cc:407] Connection header received
0807 18:17:11.866127 (+ 2115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:11.866137 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:11.866519 (+ 382us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:11.866866 (+ 347us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:17:11.868744 (+ 1878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:11.870364 (+ 1620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:11.872086 (+ 1722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:11.872727 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:11.872945 (+ 218us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:11.877081 (+ 4136us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:17:11.877088 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:17:11.877104 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:17:11.877171 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:17:11.881136 (+ 3965us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:17:11.882414 (+ 1278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:17:11.882423 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:17:11.882433 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:17:11.882656 (+ 223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:17:11.883224 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:17:11.883228 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:17:11.883232 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:17:11.883958 (+ 726us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:17:11.884146 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:11.885279 (+ 1133us) server_negotiation.cc:299] Negotiation successful
0807 18:17:11.885778 (+ 499us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":373,"thread_start_us":257,"threads_started":1}
I20260807 18:17:11.886189 11084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:11.864314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.1:60851 (local address 127.18.154.2:42929)
0807 18:17:11.865041 (+ 727us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:11.865050 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:11.865085 (+ 35us) server_negotiation.cc:407] Connection header received
0807 18:17:11.866651 (+ 1566us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:11.866660 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:11.866999 (+ 339us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:11.867310 (+ 311us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:17:11.868766 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:11.870313 (+ 1547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:11.872097 (+ 1784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:11.872717 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:11.872934 (+ 217us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:11.877367 (+ 4433us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:17:11.877377 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:17:11.877394 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:17:11.877443 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:17:11.881136 (+ 3693us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:17:11.882414 (+ 1278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:17:11.882423 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:17:11.882433 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:17:11.882655 (+ 222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:17:11.883229 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:17:11.883233 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:17:11.883236 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:17:11.883957 (+ 721us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:17:11.884146 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:11.885287 (+ 1141us) server_negotiation.cc:299] Negotiation successful
0807 18:17:11.885755 (+ 468us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":577,"thread_start_us":445,"threads_started":1}
I20260807 18:17:11.886250 11083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:11.863752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.2:42929 (local address 127.18.154.1:60851)
0807 18:17:11.864489 (+ 737us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:11.864512 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 18:17:11.864620 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:17:11.867280 (+ 2660us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:17:11.867285 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:17:11.867966 (+ 681us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:17:11.868552 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:11.868572 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:11.870509 (+ 1937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:11.870513 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:11.871942 (+ 1429us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:11.871954 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:11.872949 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:11.872959 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:11.873276 (+ 317us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:11.874170 (+ 894us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:17:11.874192 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:17:11.877091 (+ 2899us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:17:11.881328 (+ 4237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:17:11.881338 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:17:11.881355 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:17:11.882256 (+ 901us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:17:11.882792 (+ 536us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:17:11.882798 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:17:11.882801 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:17:11.883101 (+ 300us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:17:11.884124 (+ 1023us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:17:11.884132 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:17:11.884641 (+ 509us) client_negotiation.cc:769] Sending connection context
0807 18:17:11.885012 (+ 371us) client_negotiation.cc:240] Negotiation successful
0807 18:17:11.885812 (+ 800us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":604,"mutex_wait_us":487,"spinlock_wait_cycles":34944,"thread_start_us":729,"threads_started":1}
I20260807 18:17:11.886572 11081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:11.863101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.3:33869 (local address 127.18.154.1:47577)
0807 18:17:11.863553 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:11.863609 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0807 18:17:11.864223 (+ 614us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:17:11.866976 (+ 2753us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:17:11.866991 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:17:11.867889 (+ 898us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:17:11.868552 (+ 663us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:11.868571 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:11.870537 (+ 1966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:11.870541 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:11.871939 (+ 1398us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:11.871953 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:11.872979 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:11.872987 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:11.873274 (+ 287us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:11.874230 (+ 956us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:17:11.874247 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:17:11.876757 (+ 2510us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:17:11.881328 (+ 4571us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:17:11.881338 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:17:11.881355 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:17:11.882256 (+ 901us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:17:11.882792 (+ 536us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:17:11.882798 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:17:11.882802 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:17:11.883101 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:17:11.884124 (+ 1023us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:17:11.884132 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:17:11.884641 (+ 509us) client_negotiation.cc:769] Sending connection context
0807 18:17:11.885012 (+ 371us) client_negotiation.cc:240] Negotiation successful
0807 18:17:11.885676 (+ 664us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":317,"mutex_wait_us":223,"spinlock_wait_cycles":49024,"thread_start_us":213,"threads_started":1}
W20260807 18:17:11.889653 10561 leader_election.cc:341] T 97473341374741d6b3a437769d5d74a2 P b3f50a9f91964b79b66d73c38ac9ab47 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 90873f7a558149f094dd37e015a2d8f8 (127.18.154.3:33869): Not found: Tablet not found: 97473341374741d6b3a437769d5d74a2
W20260807 18:17:11.889986 10561 leader_election.cc:341] T 97473341374741d6b3a437769d5d74a2 P b3f50a9f91964b79b66d73c38ac9ab47 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3487895bdebd4925b1e0ee7b961c18d9 (127.18.154.2:42929): Not found: Tablet not found: 97473341374741d6b3a437769d5d74a2
I20260807 18:17:11.890117 10561 leader_election.cc:302] T 97473341374741d6b3a437769d5d74a2 P b3f50a9f91964b79b66d73c38ac9ab47 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3f50a9f91964b79b66d73c38ac9ab47; no voters: 3487895bdebd4925b1e0ee7b961c18d9, 90873f7a558149f094dd37e015a2d8f8
I20260807 18:17:11.890534 11080 raft_consensus.cc:2726] T 97473341374741d6b3a437769d5d74a2 P b3f50a9f91964b79b66d73c38ac9ab47 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 18:17:12.228094 11085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:12.227205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:46089 (local address 127.18.154.3:48275)
0807 18:17:12.227693 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:12.227854 (+ 161us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.62:46089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":381,"thread_start_us":242,"threads_started":1}
I20260807 18:17:12.251649 11083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:12.251212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:46089 (local address 127.18.154.1:45613)
0807 18:17:12.251409 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:12.251506 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.62:46089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260807 18:17:12.748028 11086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:12.747073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:46089 (local address 127.18.154.2:44901)
0807 18:17:12.747636 (+ 563us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:12.747766 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.62:46089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":419,"thread_start_us":326,"threads_started":1}
I20260807 18:17:13.229903 11087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:13.229127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:46089 (local address 127.18.154.3:50959)
0807 18:17:13.229650 (+ 523us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:13.229743 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.62:46089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":351,"thread_start_us":240,"threads_started":1}
I20260807 18:17:13.253518 11088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:13.252638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:46089 (local address 127.18.154.1:44327)
0807 18:17:13.253183 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:13.253300 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.62:46089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":436,"thread_start_us":267,"threads_started":1}
I20260807 18:17:13.749805 11089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:13.749014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:46089 (local address 127.18.154.2:39303)
0807 18:17:13.749487 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:13.749632 (+ 145us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.62:46089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":372,"thread_start_us":264,"threads_started":1}
I20260807 18:17:13.892894 11090 raft_consensus.cc:482] T 97473341374741d6b3a437769d5d74a2 P b3f50a9f91964b79b66d73c38ac9ab47 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:17:13.893085 11090 raft_consensus.cc:504] T 97473341374741d6b3a437769d5d74a2 P b3f50a9f91964b79b66d73c38ac9ab47 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3f50a9f91964b79b66d73c38ac9ab47" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 34909 } } peers { permanent_uuid: "90873f7a558149f094dd37e015a2d8f8" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 33869 } } peers { permanent_uuid: "3487895bdebd4925b1e0ee7b961c18d9" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 42929 } }
I20260807 18:17:13.893908 11090 leader_election.cc:288] T 97473341374741d6b3a437769d5d74a2 P b3f50a9f91964b79b66d73c38ac9ab47 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 90873f7a558149f094dd37e015a2d8f8 (127.18.154.3:33869), 3487895bdebd4925b1e0ee7b961c18d9 (127.18.154.2:42929)
W20260807 18:17:13.895035 10561 leader_election.cc:341] T 97473341374741d6b3a437769d5d74a2 P b3f50a9f91964b79b66d73c38ac9ab47 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 90873f7a558149f094dd37e015a2d8f8 (127.18.154.3:33869): Not found: Tablet not found: 97473341374741d6b3a437769d5d74a2
W20260807 18:17:13.895289 10561 leader_election.cc:341] T 97473341374741d6b3a437769d5d74a2 P b3f50a9f91964b79b66d73c38ac9ab47 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3487895bdebd4925b1e0ee7b961c18d9 (127.18.154.2:42929): Not found: Tablet not found: 97473341374741d6b3a437769d5d74a2
I20260807 18:17:13.895402 10561 leader_election.cc:302] T 97473341374741d6b3a437769d5d74a2 P b3f50a9f91964b79b66d73c38ac9ab47 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3f50a9f91964b79b66d73c38ac9ab47; no voters: 3487895bdebd4925b1e0ee7b961c18d9, 90873f7a558149f094dd37e015a2d8f8
I20260807 18:17:13.895691 11090 raft_consensus.cc:2726] T 97473341374741d6b3a437769d5d74a2 P b3f50a9f91964b79b66d73c38ac9ab47 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 18:17:14.231523 11091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:14.230841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:46089 (local address 127.18.154.3:44389)
0807 18:17:14.231278 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:14.231360 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.62:46089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":330,"thread_start_us":226,"threads_started":1}
I20260807 18:17:14.255465 11092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:14.254633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:46089 (local address 127.18.154.1:36609)
0807 18:17:14.255146 (+ 513us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:14.255276 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.62:46089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":396,"thread_start_us":205,"threads_started":1}
I20260807 18:17:14.751837 11093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:14.750772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:46089 (local address 127.18.154.2:50019)
0807 18:17:14.751498 (+ 726us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:14.751626 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.62:46089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":596,"thread_start_us":429,"threads_started":1}
I20260807 18:17:15.233347 11094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:15.232612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:46089 (local address 127.18.154.3:59241)
0807 18:17:15.233079 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:15.233166 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.62:46089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":363,"thread_start_us":243,"threads_started":1}
I20260807 18:17:15.257321 11095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:15.256434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:46089 (local address 127.18.154.1:50189)
0807 18:17:15.256925 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:15.257028 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.62:46089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":388,"thread_start_us":221,"threads_started":1}
I20260807 18:17:15.753489 11096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:15.752718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:46089 (local address 127.18.154.2:57345)
0807 18:17:15.753180 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:15.753258 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.62:46089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":356,"thread_start_us":249,"threads_started":1}
I20260807 18:17:16.104491 11097 raft_consensus.cc:482] T 97473341374741d6b3a437769d5d74a2 P b3f50a9f91964b79b66d73c38ac9ab47 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:17:16.104748 11097 raft_consensus.cc:504] T 97473341374741d6b3a437769d5d74a2 P b3f50a9f91964b79b66d73c38ac9ab47 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3f50a9f91964b79b66d73c38ac9ab47" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 34909 } } peers { permanent_uuid: "90873f7a558149f094dd37e015a2d8f8" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 33869 } } peers { permanent_uuid: "3487895bdebd4925b1e0ee7b961c18d9" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 42929 } }
I20260807 18:17:16.105479 11097 leader_election.cc:288] T 97473341374741d6b3a437769d5d74a2 P b3f50a9f91964b79b66d73c38ac9ab47 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 90873f7a558149f094dd37e015a2d8f8 (127.18.154.3:33869), 3487895bdebd4925b1e0ee7b961c18d9 (127.18.154.2:42929)
W20260807 18:17:16.106705 10561 leader_election.cc:341] T 97473341374741d6b3a437769d5d74a2 P b3f50a9f91964b79b66d73c38ac9ab47 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 90873f7a558149f094dd37e015a2d8f8 (127.18.154.3:33869): Not found: Tablet not found: 97473341374741d6b3a437769d5d74a2
W20260807 18:17:16.106950 10561 leader_election.cc:341] T 97473341374741d6b3a437769d5d74a2 P b3f50a9f91964b79b66d73c38ac9ab47 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3487895bdebd4925b1e0ee7b961c18d9 (127.18.154.2:42929): Not found: Tablet not found: 97473341374741d6b3a437769d5d74a2
I20260807 18:17:16.107082 10561 leader_election.cc:302] T 97473341374741d6b3a437769d5d74a2 P b3f50a9f91964b79b66d73c38ac9ab47 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3f50a9f91964b79b66d73c38ac9ab47; no voters: 3487895bdebd4925b1e0ee7b961c18d9, 90873f7a558149f094dd37e015a2d8f8
I20260807 18:17:16.107391 11097 raft_consensus.cc:2726] T 97473341374741d6b3a437769d5d74a2 P b3f50a9f91964b79b66d73c38ac9ab47 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 18:17:16.235378 11099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:16.234455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:46089 (local address 127.18.154.3:53079)
0807 18:17:16.235067 (+ 612us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:16.235190 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.62:46089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":486,"thread_start_us":321,"threads_started":1}
I20260807 18:17:16.259123 11100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:16.258301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:46089 (local address 127.18.154.1:60915)
0807 18:17:16.258825 (+ 524us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:16.258931 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.62:46089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":413,"thread_start_us":257,"threads_started":1}
I20260807 18:17:16.755558 11101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:16.754692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:46089 (local address 127.18.154.2:38197)
0807 18:17:16.755198 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:16.755323 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.62:46089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":373,"thread_start_us":259,"threads_started":1}
I20260807 18:17:17.237373 11102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:17.236582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:46089 (local address 127.18.154.3:35029)
0807 18:17:17.237072 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:17.237184 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.62:46089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":390,"thread_start_us":257,"threads_started":1}
I20260807 18:17:17.260751 11103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:17.259945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:46089 (local address 127.18.154.1:42435)
0807 18:17:17.260472 (+ 527us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:17.260557 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.62:46089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":427,"thread_start_us":294,"threads_started":1}
I20260807 18:17:17.757397 11104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:17.756569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:46089 (local address 127.18.154.2:49853)
0807 18:17:17.757101 (+ 532us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:17.757202 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.62:46089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":413,"thread_start_us":233,"threads_started":1}
I20260807 18:17:18.239253 11105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:18.238405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:46089 (local address 127.18.154.3:39935)
0807 18:17:18.238954 (+ 549us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:18.239046 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.62:46089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":449,"thread_start_us":255,"threads_started":1}
I20260807 18:17:18.262494 11106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:18.261749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:46089 (local address 127.18.154.1:45355)
0807 18:17:18.262215 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:18.262326 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.62:46089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":358,"thread_start_us":219,"threads_started":1}
I20260807 18:17:18.339196 11107 raft_consensus.cc:482] T 97473341374741d6b3a437769d5d74a2 P b3f50a9f91964b79b66d73c38ac9ab47 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:17:18.339438 11107 raft_consensus.cc:504] T 97473341374741d6b3a437769d5d74a2 P b3f50a9f91964b79b66d73c38ac9ab47 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3f50a9f91964b79b66d73c38ac9ab47" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 34909 } } peers { permanent_uuid: "90873f7a558149f094dd37e015a2d8f8" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 33869 } } peers { permanent_uuid: "3487895bdebd4925b1e0ee7b961c18d9" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 42929 } }
I20260807 18:17:18.340101 11107 leader_election.cc:288] T 97473341374741d6b3a437769d5d74a2 P b3f50a9f91964b79b66d73c38ac9ab47 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 90873f7a558149f094dd37e015a2d8f8 (127.18.154.3:33869), 3487895bdebd4925b1e0ee7b961c18d9 (127.18.154.2:42929)
W20260807 18:17:18.341115 10561 leader_election.cc:341] T 97473341374741d6b3a437769d5d74a2 P b3f50a9f91964b79b66d73c38ac9ab47 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 90873f7a558149f094dd37e015a2d8f8 (127.18.154.3:33869): Not found: Tablet not found: 97473341374741d6b3a437769d5d74a2
W20260807 18:17:18.341373 10561 leader_election.cc:341] T 97473341374741d6b3a437769d5d74a2 P b3f50a9f91964b79b66d73c38ac9ab47 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3487895bdebd4925b1e0ee7b961c18d9 (127.18.154.2:42929): Not found: Tablet not found: 97473341374741d6b3a437769d5d74a2
I20260807 18:17:18.341492 10561 leader_election.cc:302] T 97473341374741d6b3a437769d5d74a2 P b3f50a9f91964b79b66d73c38ac9ab47 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3f50a9f91964b79b66d73c38ac9ab47; no voters: 3487895bdebd4925b1e0ee7b961c18d9, 90873f7a558149f094dd37e015a2d8f8
I20260807 18:17:18.341796 11107 raft_consensus.cc:2726] T 97473341374741d6b3a437769d5d74a2 P b3f50a9f91964b79b66d73c38ac9ab47 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 18:17:18.759294 11108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:18.758497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:46089 (local address 127.18.154.2:39505)
0807 18:17:18.758987 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:18.759092 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.62:46089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":377,"thread_start_us":254,"threads_started":1}
I20260807 18:17:19.241125 11109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:19.240287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:46089 (local address 127.18.154.3:51277)
0807 18:17:19.240799 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:19.240907 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.62:46089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":378,"thread_start_us":245,"threads_started":1}
I20260807 18:17:19.264281 11111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:19.263433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:46089 (local address 127.18.154.1:40137)
0807 18:17:19.263980 (+ 547us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:19.264073 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.62:46089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":426,"thread_start_us":315,"threads_started":1}
I20260807 18:17:19.762900 11112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:19.761881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:46089 (local address 127.18.154.2:54097)
0807 18:17:19.762452 (+ 571us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:19.762631 (+ 179us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.62:46089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":434,"thread_start_us":277,"threads_started":1}
I20260807 18:17:20.243074 11113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:20.242210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:46089 (local address 127.18.154.3:38319)
0807 18:17:20.242760 (+ 550us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:20.242877 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.62:46089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":393,"thread_start_us":258,"threads_started":1}
I20260807 18:17:20.266242 11115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:20.265407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:46089 (local address 127.18.154.1:38347)
0807 18:17:20.265950 (+ 543us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:20.266042 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.62:46089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":440,"thread_start_us":331,"threads_started":1}
I20260807 18:17:20.765021 11116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:20.764026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:46089 (local address 127.18.154.2:48741)
0807 18:17:20.764652 (+ 626us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:20.764776 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.62:46089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":486,"thread_start_us":325,"threads_started":1}
I20260807 18:17:21.244910 11117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:21.244113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:46089 (local address 127.18.154.3:54707)
0807 18:17:21.244623 (+ 510us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:21.244714 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.62:46089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":402,"thread_start_us":283,"threads_started":1}
I20260807 18:17:21.268148 11119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:21.267312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:46089 (local address 127.18.154.1:32935)
0807 18:17:21.267869 (+ 557us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:21.267966 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.62:46089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":455,"thread_start_us":342,"threads_started":1}
I20260807 18:17:21.767146 11120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:21.766212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:46089 (local address 127.18.154.2:47961)
0807 18:17:21.766873 (+ 661us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:21.766962 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.62:46089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":542,"thread_start_us":284,"threads_started":1}
I20260807 18:17:22.246794 11121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:22.246023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:46089 (local address 127.18.154.3:43559)
0807 18:17:22.246512 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:22.246608 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.62:46089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":377,"thread_start_us":236,"threads_started":1}
I20260807 18:17:22.270045 11122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:22.269140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:46089 (local address 127.18.154.1:43927)
0807 18:17:22.269776 (+ 636us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:22.269865 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.62:46089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":366,"thread_start_us":229,"threads_started":1}
I20260807 18:17:22.769114 11123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:22.768360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:46089 (local address 127.18.154.2:43485)
0807 18:17:22.768845 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:22.768936 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.62:46089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":376,"thread_start_us":250,"threads_started":1}
I20260807 18:17:23.248440 11124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:23.247694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:46089 (local address 127.18.154.3:54453)
0807 18:17:23.248176 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:23.248263 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.62:46089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":375,"thread_start_us":246,"threads_started":1}
I20260807 18:17:23.271934 11125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:23.271065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:46089 (local address 127.18.154.1:32827)
0807 18:17:23.271631 (+ 566us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:23.271736 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.62:46089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":428,"thread_start_us":282,"threads_started":1}
I20260807 18:17:23.771112 11127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:23.770208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:46089 (local address 127.18.154.2:34677)
0807 18:17:23.770858 (+ 650us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:23.770948 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.62:46089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":546,"thread_start_us":310,"threads_started":1}
I20260807 18:17:24.250252 11128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:24.249385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:46089 (local address 127.18.154.3:38209)
0807 18:17:24.249943 (+ 558us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:24.250080 (+ 137us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.62:46089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":444,"thread_start_us":320,"threads_started":1}
I20260807 18:17:24.274089 11130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:24.273141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:46089 (local address 127.18.154.1:42791)
0807 18:17:24.273765 (+ 624us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:24.273876 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.62:46089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":478,"thread_start_us":268,"threads_started":1}
I20260807 18:17:24.773980 11133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:24.772973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:46089 (local address 127.18.154.2:50367)
0807 18:17:24.773571 (+ 598us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:24.773695 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.62:46089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":417,"thread_start_us":267,"threads_started":1}
I20260807 18:17:25.251833 11139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:25.251108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:46089 (local address 127.18.154.3:40923)
0807 18:17:25.251550 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:25.251631 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.62:46089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":236,"threads_started":1}
I20260807 18:17:25.276376 11140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:25.275463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:46089 (local address 127.18.154.1:48495)
0807 18:17:25.276077 (+ 614us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:25.276196 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.62:46089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":489,"thread_start_us":339,"threads_started":1}
I20260807 18:17:25.751571 11141 raft_consensus.cc:482] T 97473341374741d6b3a437769d5d74a2 P b3f50a9f91964b79b66d73c38ac9ab47 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:17:25.751855 11141 raft_consensus.cc:504] T 97473341374741d6b3a437769d5d74a2 P b3f50a9f91964b79b66d73c38ac9ab47 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3f50a9f91964b79b66d73c38ac9ab47" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 34909 } } peers { permanent_uuid: "90873f7a558149f094dd37e015a2d8f8" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 33869 } } peers { permanent_uuid: "3487895bdebd4925b1e0ee7b961c18d9" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 42929 } }
I20260807 18:17:25.752580 11141 leader_election.cc:288] T 97473341374741d6b3a437769d5d74a2 P b3f50a9f91964b79b66d73c38ac9ab47 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 90873f7a558149f094dd37e015a2d8f8 (127.18.154.3:33869), 3487895bdebd4925b1e0ee7b961c18d9 (127.18.154.2:42929)
W20260807 18:17:25.753722 10561 leader_election.cc:341] T 97473341374741d6b3a437769d5d74a2 P b3f50a9f91964b79b66d73c38ac9ab47 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 90873f7a558149f094dd37e015a2d8f8 (127.18.154.3:33869): Not found: Tablet not found: 97473341374741d6b3a437769d5d74a2
W20260807 18:17:25.753898 10561 leader_election.cc:341] T 97473341374741d6b3a437769d5d74a2 P b3f50a9f91964b79b66d73c38ac9ab47 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3487895bdebd4925b1e0ee7b961c18d9 (127.18.154.2:42929): Not found: Tablet not found: 97473341374741d6b3a437769d5d74a2
I20260807 18:17:25.753984 10561 leader_election.cc:302] T 97473341374741d6b3a437769d5d74a2 P b3f50a9f91964b79b66d73c38ac9ab47 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3f50a9f91964b79b66d73c38ac9ab47; no voters: 3487895bdebd4925b1e0ee7b961c18d9, 90873f7a558149f094dd37e015a2d8f8
I20260807 18:17:25.754266 11141 raft_consensus.cc:2726] T 97473341374741d6b3a437769d5d74a2 P b3f50a9f91964b79b66d73c38ac9ab47 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 18:17:25.776010 11142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:25.775215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:46089 (local address 127.18.154.2:36503)
0807 18:17:25.775687 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:25.775837 (+ 150us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.62:46089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":359,"thread_start_us":223,"threads_started":1}
I20260807 18:17:26.226970 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 10476
I20260807 18:17:26.255295 11145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:26.253109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:46089 (local address 127.18.154.3:58161)
0807 18:17:26.254938 (+ 1829us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:26.255090 (+ 152us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.62:46089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1293,"thread_start_us":460,"threads_started":1}
I20260807 18:17:26.255350 11144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:26.228572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52472 (local address 127.18.154.1:34909)
0807 18:17:26.229224 (+ 652us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:26.229232 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:26.229260 (+ 28us) server_negotiation.cc:407] Connection header received
0807 18:17:26.229323 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:26.229329 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:26.229627 (+ 298us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:26.229766 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:17:26.231393 (+ 1627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:26.232426 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:26.233719 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:26.234298 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:26.234398 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:26.247005 (+ 12607us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:17:26.247064 (+ 59us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:17:26.247082 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:17:26.247124 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:17:26.250490 (+ 3366us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:17:26.251215 (+ 725us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:17:26.251223 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:17:26.251225 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:17:26.251373 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:17:26.251756 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:17:26.251760 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:17:26.251762 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:17:26.252433 (+ 671us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:17:26.252777 (+ 344us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:26.254563 (+ 1786us) server_negotiation.cc:299] Negotiation successful
0807 18:17:26.254928 (+ 365us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":479,"thread_start_us":344,"threads_started":1}
I20260807 18:17:26.256963 10714 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:17:26.373920 11147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:26.372917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:46089 (local address 127.18.154.1:33321)
0807 18:17:26.373441 (+ 524us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:26.373640 (+ 199us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.62:46089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":404,"thread_start_us":243,"threads_started":1}
I20260807 18:17:26.374110 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 10476
I20260807 18:17:26.399823 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 10722
I20260807 18:17:26.414753 11149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:26.400712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57002 (local address 127.18.154.2:42929)
0807 18:17:26.401258 (+ 546us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:26.401263 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:26.401284 (+ 21us) server_negotiation.cc:407] Connection header received
0807 18:17:26.401346 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:26.401350 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:26.401415 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:26.401523 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:17:26.403092 (+ 1569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:26.403966 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:26.405194 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:26.405823 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:26.405955 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:26.408566 (+ 2611us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:17:26.408571 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:17:26.408575 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:17:26.408612 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:17:26.411634 (+ 3022us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:17:26.412430 (+ 796us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:17:26.412435 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:17:26.412438 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:17:26.412610 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:17:26.413055 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:17:26.413059 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:17:26.413061 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:17:26.413388 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:17:26.413734 (+ 346us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:26.414066 (+ 332us) server_negotiation.cc:299] Negotiation successful
0807 18:17:26.414348 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":433,"thread_start_us":272,"threads_started":1}
I20260807 18:17:26.415628 10889 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:17:26.501297 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 10722
I20260807 18:17:26.527377 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 10899
I20260807 18:17:26.541747 11153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:26.528471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48842 (local address 127.18.154.3:33869)
0807 18:17:26.529127 (+ 656us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:26.529132 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:26.529149 (+ 17us) server_negotiation.cc:407] Connection header received
0807 18:17:26.529204 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:26.529207 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:26.529286 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:26.529415 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:17:26.530964 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:26.531752 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:26.533039 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:26.533502 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:26.533700 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:26.536255 (+ 2555us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:17:26.536260 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:17:26.536264 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:17:26.536298 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:17:26.538962 (+ 2664us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:17:26.539654 (+ 692us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:17:26.539661 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:17:26.539664 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:17:26.539817 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:17:26.540188 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:17:26.540193 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:17:26.540197 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:17:26.540468 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:17:26.540768 (+ 300us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:26.541056 (+ 288us) server_negotiation.cc:299] Negotiation successful
0807 18:17:26.541294 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":527,"thread_start_us":352,"threads_started":1}
I20260807 18:17:26.542565 11066 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:17:26.629804 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 10899
2026-08-07T18:17:26Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (65272 ms)
[----------] 30 tests from SecurityITest (317729 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126328808105-19048-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:17:26 dist-test-slave-gkw7 krb5kdc[11159](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:17:26 dist-test-slave-gkw7 krb5kdc[11159](info): set up 2 sockets
Aug 07 18:17:26 dist-test-slave-gkw7 krb5kdc[11159](info): commencing operation
krb5kdc: starting...
W20260807 18:17:27.947669 19048 mini_kdc.cc:121] Time spent starting KDC: real 1.266s user 0.003s sys 0.039s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126328808105-19048-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126328808105-19048-0/krb5kdc/test-user.keytab.
Aug 07 18:17:28 dist-test-slave-gkw7 krb5kdc[11159](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126648, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:17:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:17:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.154.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.154.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.154.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.62.keytab.
Entry for principal kudu/127.18.154.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.62.keytab.
I20260807 18:17:28.259311 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126328808105-19048-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126328808105-19048-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126328808105-19048-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.154.62:42251
--webserver_interface=127.18.154.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.62.keytab
--principal=kudu/127.18.154.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.154.20:45675
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.154.62:42251
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126328808105-19048-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126328808105-19048-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126328808105-19048-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:17:28.623951 11174 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:17:28.624291 11174 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:17:28.624469 11174 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:17:28.631743 11174 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:17:28.631850 11174 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:17:28.631965 11174 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:17:28.632066 11174 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:17:28.632153 11174 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:17:28.642052 11174 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.154.20:45675
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126328808105-19048-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126328808105-19048-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.154.62:42251
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.62.keytab
--principal=kudu/127.18.154.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.154.62:42251
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.154.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126328808105-19048-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:17:28.643409 11174 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:17:28.644757 11174 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:17:28.657145 11187 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:17:28.657145 11185 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:17:28.657436 11182 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:17:28.658097 11174 server_base.cc:1034] running on GCE node
I20260807 18:17:28.658977 11174 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:17:28.660447 11174 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:17:28.661635 11174 hybrid_clock.cc:648] HybridClock initialized: now 1786126648661581 us; error 56 us; skew 500 ppm
I20260807 18:17:28.662007 11174 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:17:28 dist-test-slave-gkw7 krb5kdc[11159](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126648, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:17:28.665330 11174 init.cc:377] Logged in from keytab as kudu/127.18.154.62@KRBTEST.COM (short username kudu)
I20260807 18:17:28.667019 11174 webserver.cc:460] Webserver started at http://127.18.154.62:33357/ using document root <none> and password file <none>
I20260807 18:17:28.667740 11174 fs_manager.cc:359] Metadata directory not provided
I20260807 18:17:28.667891 11174 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:17:28.668216 11174 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:17:28.671389 11174 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126328808105-19048-0/minicluster-data/master-0/data/instance:
uuid: "578da878099941b09adbc32a1481f61e"
format_stamp: "Formatted at 2026-08-07 18:17:28 on dist-test-slave-gkw7"
server_key: "09c24a8ed17c9f81d242fd6ed8e679da"
server_key_iv: "c0f4de32031efa941b77f2a05f6b9dbb"
server_key_version: "encryptionkey@0"
I20260807 18:17:28.672183 11174 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126328808105-19048-0/minicluster-data/master-0/wal/instance:
uuid: "578da878099941b09adbc32a1481f61e"
format_stamp: "Formatted at 2026-08-07 18:17:28 on dist-test-slave-gkw7"
server_key: "09c24a8ed17c9f81d242fd6ed8e679da"
server_key_iv: "c0f4de32031efa941b77f2a05f6b9dbb"
server_key_version: "encryptionkey@0"
I20260807 18:17:28.677939 11174 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260807 18:17:28.681502 11197 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:17:28.682971 11174 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 18:17:28.683172 11174 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126328808105-19048-0/minicluster-data/master-0/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126328808105-19048-0/minicluster-data/master-0/wal
uuid: "578da878099941b09adbc32a1481f61e"
format_stamp: "Formatted at 2026-08-07 18:17:28 on dist-test-slave-gkw7"
server_key: "09c24a8ed17c9f81d242fd6ed8e679da"
server_key_iv: "c0f4de32031efa941b77f2a05f6b9dbb"
server_key_version: "encryptionkey@0"
I20260807 18:17:28.683398 11174 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126328808105-19048-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126328808105-19048-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126328808105-19048-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:17:28.699604 11174 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:17:28.704082 11174 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:17:28.704402 11174 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:17:28.724093 11174 rpc_server.cc:307] RPC server started. Bound to: 127.18.154.62:42251
I20260807 18:17:28.724105 11274 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.154.62:42251 every 8 connection(s)
I20260807 18:17:28.725633 11174 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
I20260807 18:17:28.729617 11275 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:17:28.735098 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 11174
I20260807 18:17:28.735361 19048 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126328808105-19048-0/minicluster-data/master-0/wal/instance
I20260807 18:17:28.736182 19048 external_mini_cluster.cc:1428] Setting key 23e860a4fb56b5abf868d744f2cc53f0
I20260807 18:17:28.741747 11275 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 578da878099941b09adbc32a1481f61e: Bootstrap starting.
I20260807 18:17:28.745831 11275 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 578da878099941b09adbc32a1481f61e: Neither blocks nor log segments found. Creating new log.
I20260807 18:17:28.747196 11275 log.cc:824] T 00000000000000000000000000000000 P 578da878099941b09adbc32a1481f61e: Log is configured to *not* fsync() on all Append() calls
I20260807 18:17:28.751267 11275 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 578da878099941b09adbc32a1481f61e: No bootstrap required, opened a new log
Aug 07 18:17:28 dist-test-slave-gkw7 krb5kdc[11159](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126648, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.154.62@KRBTEST.COM
I20260807 18:17:28.759582 11275 raft_consensus.cc:348] T 00000000000000000000000000000000 P 578da878099941b09adbc32a1481f61e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "578da878099941b09adbc32a1481f61e" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 42251 } }
I20260807 18:17:28.760015 11275 raft_consensus.cc:374] T 00000000000000000000000000000000 P 578da878099941b09adbc32a1481f61e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:17:28.760147 11275 raft_consensus.cc:729] T 00000000000000000000000000000000 P 578da878099941b09adbc32a1481f61e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 578da878099941b09adbc32a1481f61e, State: Initialized, Role: FOLLOWER
I20260807 18:17:28.760874 11275 consensus_queue.cc:260] T 00000000000000000000000000000000 P 578da878099941b09adbc32a1481f61e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "578da878099941b09adbc32a1481f61e" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 42251 } }
I20260807 18:17:28.761220 11275 raft_consensus.cc:388] T 00000000000000000000000000000000 P 578da878099941b09adbc32a1481f61e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:17:28.761382 11275 raft_consensus.cc:482] T 00000000000000000000000000000000 P 578da878099941b09adbc32a1481f61e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:17:28.761602 11275 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 578da878099941b09adbc32a1481f61e [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:17:28.762465 11278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:28.738390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40310 (local address 127.18.154.62:42251)
0807 18:17:28.739300 (+ 910us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:28.739355 (+ 55us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:28.739406 (+ 51us) server_negotiation.cc:407] Connection header received
0807 18:17:28.741158 (+ 1752us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:28.741184 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:28.741677 (+ 493us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:28.742101 (+ 424us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:17:28.744031 (+ 1930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:28.745370 (+ 1339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:28.747475 (+ 2105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:28.748618 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:28.748831 (+ 213us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:28.754309 (+ 5478us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:17:28.754318 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:17:28.754338 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:17:28.754382 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:17:28.757790 (+ 3408us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:17:28.758589 (+ 799us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:17:28.758597 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:17:28.758604 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:17:28.758788 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:17:28.759217 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:17:28.759222 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:17:28.759225 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:17:28.759815 (+ 590us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:17:28.760137 (+ 322us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:28.760693 (+ 556us) server_negotiation.cc:299] Negotiation successful
0807 18:17:28.761128 (+ 435us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":656,"thread_start_us":421,"threads_started":1}
I20260807 18:17:28.764206 11275 raft_consensus.cc:504] T 00000000000000000000000000000000 P 578da878099941b09adbc32a1481f61e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "578da878099941b09adbc32a1481f61e" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 42251 } }
I20260807 18:17:28.764750 11275 leader_election.cc:302] T 00000000000000000000000000000000 P 578da878099941b09adbc32a1481f61e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 578da878099941b09adbc32a1481f61e; no voters:
I20260807 18:17:28.765497 11275 leader_election.cc:288] T 00000000000000000000000000000000 P 578da878099941b09adbc32a1481f61e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:17:28.765667 11282 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 578da878099941b09adbc32a1481f61e [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:17:28.766434 11282 raft_consensus.cc:686] T 00000000000000000000000000000000 P 578da878099941b09adbc32a1481f61e [term 1 LEADER]: Becoming Leader. State: Replica: 578da878099941b09adbc32a1481f61e, State: Running, Role: LEADER
I20260807 18:17:28.767084 11282 consensus_queue.cc:237] T 00000000000000000000000000000000 P 578da878099941b09adbc32a1481f61e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "578da878099941b09adbc32a1481f61e" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 42251 } }
I20260807 18:17:28.768000 11275 sys_catalog.cc:564] T 00000000000000000000000000000000 P 578da878099941b09adbc32a1481f61e [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:17:28.770076 11285 sys_catalog.cc:455] T 00000000000000000000000000000000 P 578da878099941b09adbc32a1481f61e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 578da878099941b09adbc32a1481f61e. Latest consensus state: current_term: 1 leader_uuid: "578da878099941b09adbc32a1481f61e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "578da878099941b09adbc32a1481f61e" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 42251 } } }
I20260807 18:17:28.770496 11285 sys_catalog.cc:458] T 00000000000000000000000000000000 P 578da878099941b09adbc32a1481f61e [sys.catalog]: This master's current role is: LEADER
I20260807 18:17:28.771287 11283 sys_catalog.cc:455] T 00000000000000000000000000000000 P 578da878099941b09adbc32a1481f61e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "578da878099941b09adbc32a1481f61e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "578da878099941b09adbc32a1481f61e" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 42251 } } }
I20260807 18:17:28.771561 11293 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:17:28.771678 11283 sys_catalog.cc:458] T 00000000000000000000000000000000 P 578da878099941b09adbc32a1481f61e [sys.catalog]: This master's current role is: LEADER
I20260807 18:17:28.778628 11293 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:17:28.789322 11293 catalog_manager.cc:1347] Generated new cluster ID: 45bf14fe288344f1a27e8eaf043bc29a
I20260807 18:17:28.789486 11293 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:17:28.808516 11293 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:17:28.810130 11293 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:17:28.818521 11293 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 578da878099941b09adbc32a1481f61e: Generated new TSK 0
I20260807 18:17:28.819708 11293 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.154.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.154.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.154.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.1.keytab.
Entry for principal kudu/127.18.154.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.1.keytab.
I20260807 18:17:28.906054 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126328808105-19048-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126328808105-19048-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126328808105-19048-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126328808105-19048-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.154.1:0
--local_ip_for_outbound_sockets=127.18.154.1
--webserver_interface=127.18.154.1
--webserver_port=0
--tserver_master_addrs=127.18.154.62:42251
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.1.keytab
--principal=kudu/127.18.154.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.154.20:45675
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126328808105-19048-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126328808105-19048-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126328808105-19048-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:17:29.268782 11313 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:17:29.269166 11313 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:17:29.269390 11313 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:17:29.280331 11313 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:17:29.280489 11313 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:17:29.280666 11313 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.154.1
I20260807 18:17:29.291155 11313 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.154.20:45675
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126328808105-19048-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126328808105-19048-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.1.keytab
--principal=kudu/127.18.154.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.154.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126328808105-19048-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.154.1
--webserver_port=0
--tserver_master_addrs=127.18.154.62:42251
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.154.1
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126328808105-19048-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:17:29.292598 11313 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:17:29.294059 11313 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:17:29.307716 11319 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:17:29.307821 11323 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:17:29.307840 11320 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:17:29.308511 11313 server_base.cc:1034] running on GCE node
I20260807 18:17:29.309195 11313 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:17:29.310230 11313 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:17:29.311446 11313 hybrid_clock.cc:648] HybridClock initialized: now 1786126649311391 us; error 40 us; skew 500 ppm
I20260807 18:17:29.311751 11313 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:17:29 dist-test-slave-gkw7 krb5kdc[11159](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126649, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:17:29.315069 11313 init.cc:377] Logged in from keytab as kudu/127.18.154.1@KRBTEST.COM (short username kudu)
I20260807 18:17:29.316636 11313 webserver.cc:460] Webserver started at http://127.18.154.1:42547/ using document root <none> and password file <none>
I20260807 18:17:29.317378 11313 fs_manager.cc:359] Metadata directory not provided
I20260807 18:17:29.317520 11313 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:17:29.317914 11313 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:17:29.320966 11313 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126328808105-19048-0/minicluster-data/ts-0/data/instance:
uuid: "fd77991ca716451689a298661577abf0"
format_stamp: "Formatted at 2026-08-07 18:17:29 on dist-test-slave-gkw7"
server_key: "e60b11a75a3b074d4a635d4197eb204d"
server_key_iv: "b74e7c89596321697d41c0e9729e9f94"
server_key_version: "encryptionkey@0"
I20260807 18:17:29.321851 11313 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126328808105-19048-0/minicluster-data/ts-0/wal/instance:
uuid: "fd77991ca716451689a298661577abf0"
format_stamp: "Formatted at 2026-08-07 18:17:29 on dist-test-slave-gkw7"
server_key: "e60b11a75a3b074d4a635d4197eb204d"
server_key_iv: "b74e7c89596321697d41c0e9729e9f94"
server_key_version: "encryptionkey@0"
I20260807 18:17:29.327435 11313 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260807 18:17:29.331144 11335 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:17:29.332679 11313 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 18:17:29.332947 11313 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126328808105-19048-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126328808105-19048-0/minicluster-data/ts-0/wal
uuid: "fd77991ca716451689a298661577abf0"
format_stamp: "Formatted at 2026-08-07 18:17:29 on dist-test-slave-gkw7"
server_key: "e60b11a75a3b074d4a635d4197eb204d"
server_key_iv: "b74e7c89596321697d41c0e9729e9f94"
server_key_version: "encryptionkey@0"
I20260807 18:17:29.333180 11313 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126328808105-19048-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126328808105-19048-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126328808105-19048-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:17:29.350386 11313 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:17:29.354874 11313 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:17:29.355194 11313 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:17:29.356274 11313 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:17:29.357787 11313 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:17:29.357865 11313 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:17:29.358019 11313 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:17:29.358096 11313 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:17:29.394277 11313 rpc_server.cc:307] RPC server started. Bound to: 127.18.154.1:35983
I20260807 18:17:29.394333 11488 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.154.1:35983 every 8 connection(s)
I20260807 18:17:29.395762 11313 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126328808105-19048-0/minicluster-data/ts-0/data/info.pb
I20260807 18:17:29.398840 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 11313
I20260807 18:17:29.399061 19048 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126328808105-19048-0/minicluster-data/ts-0/wal/instance
I20260807 18:17:29.399981 19048 external_mini_cluster.cc:1428] Setting key cc213b8d70112d676049776bbdc10a67
Aug 07 18:17:29 dist-test-slave-gkw7 krb5kdc[11159](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126649, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.1@KRBTEST.COM for kudu/127.18.154.62@KRBTEST.COM
I20260807 18:17:29.416896 11493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:29.398516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.1:39957 (local address 127.18.154.62:42251)
0807 18:17:29.399033 (+ 517us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:29.399039 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:29.400314 (+ 1275us) server_negotiation.cc:407] Connection header received
0807 18:17:29.402806 (+ 2492us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:29.402811 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:29.402901 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:29.403018 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:17:29.404566 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:29.405288 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:29.406960 (+ 1672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:29.407429 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:29.407567 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:29.411142 (+ 3575us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:17:29.411147 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:17:29.411150 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:17:29.411189 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:17:29.413795 (+ 2606us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:17:29.414515 (+ 720us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:17:29.414519 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:17:29.414522 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:17:29.414636 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:17:29.415182 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:17:29.415185 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:17:29.415187 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:17:29.415470 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:17:29.415588 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:29.416468 (+ 880us) server_negotiation.cc:299] Negotiation successful
0807 18:17:29.416637 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":364,"thread_start_us":260,"threads_started":1}
I20260807 18:17:29.418490 11495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:29.399276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:42251 (local address 127.18.154.1:39957)
0807 18:17:29.400135 (+ 859us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:29.400185 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0807 18:17:29.402531 (+ 2346us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:17:29.403277 (+ 746us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:17:29.403285 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:17:29.403809 (+ 524us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:17:29.404390 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:29.404409 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:29.405468 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:29.405472 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:29.406814 (+ 1342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:29.406826 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:29.407678 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:29.407685 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:29.407936 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:29.408630 (+ 694us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:17:29.408653 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:17:29.410919 (+ 2266us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:17:29.413934 (+ 3015us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:17:29.413943 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:17:29.413964 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:17:29.414380 (+ 416us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:17:29.414747 (+ 367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:17:29.414753 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:17:29.414756 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:17:29.415009 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:17:29.415605 (+ 596us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:17:29.415616 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:17:29.416068 (+ 452us) client_negotiation.cc:769] Sending connection context
0807 18:17:29.416363 (+ 295us) client_negotiation.cc:240] Negotiation successful
0807 18:17:29.416977 (+ 614us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":555,"thread_start_us":340,"threads_started":1}
I20260807 18:17:29.420118 11490 heartbeater.cc:347] Connected to a master server at 127.18.154.62:42251
I20260807 18:17:29.420454 11490 heartbeater.cc:464] Registering TS with master...
I20260807 18:17:29.421371 11490 heartbeater.cc:511] Master 127.18.154.62:42251 requested a full tablet report, sending...
I20260807 18:17:29.423691 11227 ts_manager.cc:194] Registered new tserver with Master: fd77991ca716451689a298661577abf0 (127.18.154.1:35983)
I20260807 18:17:29.425207 11227 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.154.1@KRBTEST.COM'} at 127.18.154.1:39957
WARNING: no policy specified for kudu/127.18.154.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.154.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.154.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.2.keytab.
Entry for principal kudu/127.18.154.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.2.keytab.
I20260807 18:17:29.476713 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126328808105-19048-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126328808105-19048-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126328808105-19048-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126328808105-19048-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.154.2:0
--local_ip_for_outbound_sockets=127.18.154.2
--webserver_interface=127.18.154.2
--webserver_port=0
--tserver_master_addrs=127.18.154.62:42251
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.2.keytab
--principal=kudu/127.18.154.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.154.20:45675
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126328808105-19048-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126328808105-19048-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126328808105-19048-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:17:29.834378 11499 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:17:29.834740 11499 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:17:29.834924 11499 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:17:29.842655 11499 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:17:29.842813 11499 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:17:29.843008 11499 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.154.2
I20260807 18:17:29.853034 11499 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.154.20:45675
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126328808105-19048-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126328808105-19048-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.2.keytab
--principal=kudu/127.18.154.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.154.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126328808105-19048-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.154.2
--webserver_port=0
--tserver_master_addrs=127.18.154.62:42251
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.154.2
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126328808105-19048-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:17:29.854507 11499 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:17:29.855857 11499 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:17:29.869547 11514 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:17:29.869582 11515 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:17:29.870132 11518 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:17:29.870752 11499 server_base.cc:1034] running on GCE node
I20260807 18:17:29.871460 11499 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:17:29.872432 11499 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:17:29.873642 11499 hybrid_clock.cc:648] HybridClock initialized: now 1786126649873579 us; error 70 us; skew 500 ppm
I20260807 18:17:29.874007 11499 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:17:29 dist-test-slave-gkw7 krb5kdc[11159](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126649, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:17:29.877420 11499 init.cc:377] Logged in from keytab as kudu/127.18.154.2@KRBTEST.COM (short username kudu)
I20260807 18:17:29.878998 11499 webserver.cc:460] Webserver started at http://127.18.154.2:46507/ using document root <none> and password file <none>
I20260807 18:17:29.879776 11499 fs_manager.cc:359] Metadata directory not provided
I20260807 18:17:29.879930 11499 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:17:29.880286 11499 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:17:29.883343 11499 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126328808105-19048-0/minicluster-data/ts-1/data/instance:
uuid: "bf1991da7e974a75b86b39d7abef9475"
format_stamp: "Formatted at 2026-08-07 18:17:29 on dist-test-slave-gkw7"
server_key: "220d89514fe9a59231feb805c106234c"
server_key_iv: "3c7bf80b19cad935d5ce464ed8ad093e"
server_key_version: "encryptionkey@0"
I20260807 18:17:29.884192 11499 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126328808105-19048-0/minicluster-data/ts-1/wal/instance:
uuid: "bf1991da7e974a75b86b39d7abef9475"
format_stamp: "Formatted at 2026-08-07 18:17:29 on dist-test-slave-gkw7"
server_key: "220d89514fe9a59231feb805c106234c"
server_key_iv: "3c7bf80b19cad935d5ce464ed8ad093e"
server_key_version: "encryptionkey@0"
I20260807 18:17:29.889988 11499 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260807 18:17:29.893483 11526 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:17:29.894887 11499 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 18:17:29.895076 11499 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126328808105-19048-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126328808105-19048-0/minicluster-data/ts-1/wal
uuid: "bf1991da7e974a75b86b39d7abef9475"
format_stamp: "Formatted at 2026-08-07 18:17:29 on dist-test-slave-gkw7"
server_key: "220d89514fe9a59231feb805c106234c"
server_key_iv: "3c7bf80b19cad935d5ce464ed8ad093e"
server_key_version: "encryptionkey@0"
I20260807 18:17:29.895287 11499 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126328808105-19048-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126328808105-19048-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126328808105-19048-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:17:29.915824 11499 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:17:29.920563 11499 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:17:29.920879 11499 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:17:29.921973 11499 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:17:29.923333 11499 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:17:29.923442 11499 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:17:29.923595 11499 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:17:29.923669 11499 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:17:29.958017 11499 rpc_server.cc:307] RPC server started. Bound to: 127.18.154.2:39673
I20260807 18:17:29.958063 11677 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.154.2:39673 every 8 connection(s)
I20260807 18:17:29.959429 11499 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126328808105-19048-0/minicluster-data/ts-1/data/info.pb
I20260807 18:17:29.960103 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 11499
I20260807 18:17:29.960325 19048 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126328808105-19048-0/minicluster-data/ts-1/wal/instance
I20260807 18:17:29.961032 19048 external_mini_cluster.cc:1428] Setting key 0827a37b65c38fb81bd4922feb2c0966
Aug 07 18:17:29 dist-test-slave-gkw7 krb5kdc[11159](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126649, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.2@KRBTEST.COM for kudu/127.18.154.62@KRBTEST.COM
I20260807 18:17:29.986583 11680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:29.962463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.2:58957 (local address 127.18.154.62:42251)
0807 18:17:29.962983 (+ 520us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:29.962989 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:29.964263 (+ 1274us) server_negotiation.cc:407] Connection header received
0807 18:17:29.967274 (+ 3011us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:29.967278 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:29.967346 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:29.967440 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:17:29.969694 (+ 2254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:29.970471 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:29.973072 (+ 2601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:29.973574 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:29.973688 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:29.978569 (+ 4881us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:17:29.978580 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:17:29.978589 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:17:29.978641 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:17:29.982555 (+ 3914us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:17:29.983488 (+ 933us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:17:29.983497 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:17:29.983504 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:17:29.983691 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:17:29.984176 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:17:29.984185 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:17:29.984192 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:17:29.984683 (+ 491us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:17:29.984860 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:29.985674 (+ 814us) server_negotiation.cc:299] Negotiation successful
0807 18:17:29.985972 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":393,"thread_start_us":254,"threads_started":1}
I20260807 18:17:29.987373 11681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:29.963198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:42251 (local address 127.18.154.2:58957)
0807 18:17:29.964062 (+ 864us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:29.964125 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0807 18:17:29.966951 (+ 2826us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:17:29.967755 (+ 804us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:17:29.967777 (+ 22us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:17:29.968582 (+ 805us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:17:29.969469 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:29.969498 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:29.970629 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:29.970638 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:29.972921 (+ 2283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:29.972940 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:29.973834 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:29.973845 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:29.974218 (+ 373us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:29.974986 (+ 768us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:17:29.975029 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:17:29.978258 (+ 3229us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:17:29.982754 (+ 4496us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:17:29.982762 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:17:29.982778 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:17:29.983312 (+ 534us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:17:29.983813 (+ 501us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:17:29.983817 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:17:29.983819 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:17:29.984046 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:17:29.984844 (+ 798us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:17:29.984851 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:17:29.985204 (+ 353us) client_negotiation.cc:769] Sending connection context
0807 18:17:29.985473 (+ 269us) client_negotiation.cc:240] Negotiation successful
0807 18:17:29.986026 (+ 553us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":532,"thread_start_us":306,"threads_started":1}
I20260807 18:17:29.989065 11678 heartbeater.cc:347] Connected to a master server at 127.18.154.62:42251
I20260807 18:17:29.989451 11678 heartbeater.cc:464] Registering TS with master...
I20260807 18:17:29.990473 11678 heartbeater.cc:511] Master 127.18.154.62:42251 requested a full tablet report, sending...
I20260807 18:17:29.992324 11227 ts_manager.cc:194] Registered new tserver with Master: bf1991da7e974a75b86b39d7abef9475 (127.18.154.2:39673)
I20260807 18:17:29.993181 11227 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.154.2@KRBTEST.COM'} at 127.18.154.2:58957
WARNING: no policy specified for kudu/127.18.154.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.154.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.154.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.3.keytab.
Entry for principal kudu/127.18.154.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.3.keytab.
I20260807 18:17:30.037034 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126328808105-19048-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126328808105-19048-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126328808105-19048-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126328808105-19048-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.154.3:0
--local_ip_for_outbound_sockets=127.18.154.3
--webserver_interface=127.18.154.3
--webserver_port=0
--tserver_master_addrs=127.18.154.62:42251
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.3.keytab
--principal=kudu/127.18.154.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.154.20:45675
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126328808105-19048-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126328808105-19048-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126328808105-19048-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:17:30.384235 11687 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:17:30.384603 11687 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:17:30.384781 11687 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:17:30.392513 11687 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:17:30.392690 11687 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:17:30.392868 11687 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.154.3
I20260807 18:17:30.403429 11687 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.154.20:45675
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126328808105-19048-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126328808105-19048-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.3.keytab
--principal=kudu/127.18.154.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.154.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126328808105-19048-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.154.3
--webserver_port=0
--tserver_master_addrs=127.18.154.62:42251
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.154.3
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126328808105-19048-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:17:30.404727 11687 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:17:30.406121 11687 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:17:30.419914 11694 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:17:30.419957 11695 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:17:30.419874 11701 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:17:30.423202 11687 server_base.cc:1034] running on GCE node
I20260807 18:17:30.423882 11687 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:17:30.424839 11687 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:17:30.426069 11687 hybrid_clock.cc:648] HybridClock initialized: now 1786126650426015 us; error 48 us; skew 500 ppm
I20260807 18:17:30.426399 11687 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:17:30 dist-test-slave-gkw7 krb5kdc[11159](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126650, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:17:30.429159 11490 heartbeater.cc:503] Master 127.18.154.62:42251 was elected leader, sending a full tablet report...
I20260807 18:17:30.429741 11687 init.cc:377] Logged in from keytab as kudu/127.18.154.3@KRBTEST.COM (short username kudu)
I20260807 18:17:30.432040 11687 webserver.cc:460] Webserver started at http://127.18.154.3:36669/ using document root <none> and password file <none>
I20260807 18:17:30.432732 11687 fs_manager.cc:359] Metadata directory not provided
I20260807 18:17:30.432834 11687 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:17:30.433094 11687 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:17:30.436002 11687 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126328808105-19048-0/minicluster-data/ts-2/data/instance:
uuid: "7bcf4ff6892d451faeadaa7f385f9f26"
format_stamp: "Formatted at 2026-08-07 18:17:30 on dist-test-slave-gkw7"
server_key: "562292cbd12bad9ab8c55d1367f93d29"
server_key_iv: "302076f1521f0b1db8ff8712a7d315e8"
server_key_version: "encryptionkey@0"
I20260807 18:17:30.436878 11687 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126328808105-19048-0/minicluster-data/ts-2/wal/instance:
uuid: "7bcf4ff6892d451faeadaa7f385f9f26"
format_stamp: "Formatted at 2026-08-07 18:17:30 on dist-test-slave-gkw7"
server_key: "562292cbd12bad9ab8c55d1367f93d29"
server_key_iv: "302076f1521f0b1db8ff8712a7d315e8"
server_key_version: "encryptionkey@0"
I20260807 18:17:30.442422 11687 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260807 18:17:30.445945 11711 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:17:30.447285 11687 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260807 18:17:30.447466 11687 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126328808105-19048-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126328808105-19048-0/minicluster-data/ts-2/wal
uuid: "7bcf4ff6892d451faeadaa7f385f9f26"
format_stamp: "Formatted at 2026-08-07 18:17:30 on dist-test-slave-gkw7"
server_key: "562292cbd12bad9ab8c55d1367f93d29"
server_key_iv: "302076f1521f0b1db8ff8712a7d315e8"
server_key_version: "encryptionkey@0"
I20260807 18:17:30.447678 11687 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126328808105-19048-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126328808105-19048-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126328808105-19048-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:17:30.470063 11687 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:17:30.474761 11687 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:17:30.475096 11687 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:17:30.476244 11687 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:17:30.477707 11687 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:17:30.477815 11687 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:17:30.477967 11687 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:17:30.478041 11687 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:17:30.513887 11687 rpc_server.cc:307] RPC server started. Bound to: 127.18.154.3:36677
I20260807 18:17:30.513926 11870 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.154.3:36677 every 8 connection(s)
I20260807 18:17:30.515319 11687 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126328808105-19048-0/minicluster-data/ts-2/data/info.pb
I20260807 18:17:30.518748 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 11687
I20260807 18:17:30.518965 19048 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126328808105-19048-0/minicluster-data/ts-2/wal/instance
I20260807 18:17:30.519668 19048 external_mini_cluster.cc:1428] Setting key 7c08b8e1fb0187b092ef77394dd31703
Aug 07 18:17:30 dist-test-slave-gkw7 krb5kdc[11159](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126650, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.3@KRBTEST.COM for kudu/127.18.154.62@KRBTEST.COM
I20260807 18:17:30.536276 11873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:30.518074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.3:33705 (local address 127.18.154.62:42251)
0807 18:17:30.518619 (+ 545us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:30.518624 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:30.519734 (+ 1110us) server_negotiation.cc:407] Connection header received
0807 18:17:30.522045 (+ 2311us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:30.522051 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:30.522166 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:30.522283 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:17:30.523951 (+ 1668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:30.524763 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:30.526422 (+ 1659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:30.526835 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:30.526973 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:30.530371 (+ 3398us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:17:30.530376 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:17:30.530380 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:17:30.530413 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:17:30.532895 (+ 2482us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:17:30.533837 (+ 942us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:17:30.533841 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:17:30.533844 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:17:30.533967 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:17:30.534525 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:17:30.534530 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:17:30.534532 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:17:30.534867 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:17:30.535004 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:30.535813 (+ 809us) server_negotiation.cc:299] Negotiation successful
0807 18:17:30.535982 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":446,"thread_start_us":311,"threads_started":1}
I20260807 18:17:30.537314 11875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:30.518720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:42251 (local address 127.18.154.3:33705)
0807 18:17:30.519569 (+ 849us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:30.519606 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 18:17:30.521722 (+ 2116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:17:30.522543 (+ 821us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:17:30.522551 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:17:30.523206 (+ 655us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:17:30.523780 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:30.523797 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:30.524921 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:30.524933 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:30.526267 (+ 1334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:30.526280 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:30.527067 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:30.527077 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:30.527319 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:30.527941 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:17:30.527968 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:17:30.530169 (+ 2201us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:17:30.533028 (+ 2859us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:17:30.533035 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:17:30.533047 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:17:30.533684 (+ 637us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:17:30.534108 (+ 424us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:17:30.534112 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:17:30.534114 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:17:30.534397 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:17:30.535001 (+ 604us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:17:30.535009 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:17:30.535422 (+ 413us) client_negotiation.cc:769] Sending connection context
0807 18:17:30.535688 (+ 266us) client_negotiation.cc:240] Negotiation successful
0807 18:17:30.536080 (+ 392us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":594,"thread_start_us":360,"threads_started":1}
I20260807 18:17:30.538777 11871 heartbeater.cc:347] Connected to a master server at 127.18.154.62:42251
I20260807 18:17:30.539091 11871 heartbeater.cc:464] Registering TS with master...
I20260807 18:17:30.539835 11871 heartbeater.cc:511] Master 127.18.154.62:42251 requested a full tablet report, sending...
I20260807 18:17:30.541286 11227 ts_manager.cc:194] Registered new tserver with Master: 7bcf4ff6892d451faeadaa7f385f9f26 (127.18.154.3:36677)
I20260807 18:17:30.542186 11227 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.154.3@KRBTEST.COM'} at 127.18.154.3:33705
I20260807 18:17:30.547748 19048 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 18:17:30.565397 11873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:30.552858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40320 (local address 127.18.154.62:42251)
0807 18:17:30.553035 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:30.553039 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:30.553642 (+ 603us) server_negotiation.cc:407] Connection header received
0807 18:17:30.553701 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:30.553704 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:30.553769 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:30.553845 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:17:30.555359 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:30.556086 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:30.557440 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:30.557916 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:30.558012 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:30.560349 (+ 2337us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:17:30.560353 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:17:30.560356 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:17:30.560387 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:17:30.563099 (+ 2712us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:17:30.563685 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:17:30.563688 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:17:30.563691 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:17:30.563815 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:17:30.564141 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:17:30.564145 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:17:30.564146 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:17:30.564465 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:17:30.564645 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:30.564965 (+ 320us) server_negotiation.cc:299] Negotiation successful
0807 18:17:30.565130 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 18:17:30.569602 11227 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40320:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 18:17:30.576750 11227 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:17:30.614926 11898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:30.591040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54374 (local address 127.18.154.2:39673)
0807 18:17:30.605251 (+ 14211us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:30.605264 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:30.605962 (+ 698us) server_negotiation.cc:407] Connection header received
0807 18:17:30.606055 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:30.606074 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:30.606279 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:30.606455 (+ 176us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:17:30.607479 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:30.608867 (+ 1388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:30.611982 (+ 3115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:30.613501 (+ 1519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:30.613680 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:30.613875 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:30.614385 (+ 510us) server_negotiation.cc:299] Negotiation successful
0807 18:17:30.614558 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14024,"thread_start_us":13876,"threads_started":1}
I20260807 18:17:30.615012 11893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:30.590635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.2:39673 (local address 127.0.0.1:54374)
0807 18:17:30.605691 (+ 15056us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:30.605716 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0807 18:17:30.605844 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:17:30.606544 (+ 700us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:17:30.606555 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:17:30.606604 (+ 49us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:17:30.607209 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:30.607226 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:30.609064 (+ 1838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:30.609071 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:30.611802 (+ 2731us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:30.611823 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:30.614008 (+ 2185us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:30.614013 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:30.614155 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:30.614192 (+ 37us) client_negotiation.cc:769] Sending connection context
0807 18:17:30.614328 (+ 136us) client_negotiation.cc:240] Negotiation successful
0807 18:17:30.614545 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14930,"mutex_wait_us":106,"thread_start_us":12787,"threads_started":1}
I20260807 18:17:30.614991 11895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:30.590429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60844 (local address 127.18.154.1:35983)
0807 18:17:30.605721 (+ 15292us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:30.605730 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:30.605765 (+ 35us) server_negotiation.cc:407] Connection header received
0807 18:17:30.605874 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:30.605885 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:30.606177 (+ 292us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:30.606433 (+ 256us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:17:30.607468 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:30.609106 (+ 1638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:30.612301 (+ 3195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:30.613488 (+ 1187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:30.613668 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:30.613911 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:30.614505 (+ 594us) server_negotiation.cc:299] Negotiation successful
0807 18:17:30.614674 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15080,"thread_start_us":14906,"threads_started":1}
I20260807 18:17:30.615902 11896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:30.589771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.1:35983 (local address 127.0.0.1:60844)
0807 18:17:30.603084 (+ 13313us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:30.603120 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0807 18:17:30.603306 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:17:30.606716 (+ 3410us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:17:30.606724 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:17:30.606758 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:17:30.607232 (+ 474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:30.607247 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:30.609329 (+ 2082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:30.609333 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:30.612122 (+ 2789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:30.612132 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:30.614082 (+ 1950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:30.614089 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:30.614276 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:30.614309 (+ 33us) client_negotiation.cc:769] Sending connection context
0807 18:17:30.614396 (+ 87us) client_negotiation.cc:240] Negotiation successful
0807 18:17:30.615492 (+ 1096us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13115,"mutex_wait_us":1703,"thread_start_us":15841,"threads_started":1}
I20260807 18:17:30.618422 11890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:30.589002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.3:36677 (local address 127.0.0.1:33264)
0807 18:17:30.590196 (+ 1194us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:30.590262 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0807 18:17:30.590442 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:17:30.608999 (+ 18557us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:17:30.609004 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:17:30.609035 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:17:30.611092 (+ 2057us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:30.611106 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:30.613015 (+ 1909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:30.613020 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:30.615692 (+ 2672us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:30.615709 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:30.617700 (+ 1991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:30.617706 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:30.617875 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:30.617908 (+ 33us) client_negotiation.cc:769] Sending connection context
0807 18:17:30.617995 (+ 87us) client_negotiation.cc:240] Negotiation successful
0807 18:17:30.618119 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":963,"thread_start_us":813,"threads_started":1}
I20260807 18:17:30.618685 11892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:30.589597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33264 (local address 127.18.154.3:36677)
0807 18:17:30.608401 (+ 18804us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:30.608409 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:30.608441 (+ 32us) server_negotiation.cc:407] Connection header received
0807 18:17:30.608538 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:30.608546 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:30.608777 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:30.608965 (+ 188us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:17:30.611328 (+ 2363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:30.612842 (+ 1514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:30.615906 (+ 3064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:30.617435 (+ 1529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:30.617598 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:30.617852 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:30.618168 (+ 316us) server_negotiation.cc:299] Negotiation successful
0807 18:17:30.618335 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18629,"thread_start_us":18445,"threads_started":1}
I20260807 18:17:30.619190 11586 tablet_service.cc:1467] Processing CreateTablet for tablet 11d032a472f147d9a7a9a83a6b6f51fc (DEFAULT_TABLE table=test-table [id=396fb4fe48bf4c9bb4b02030131b9b16]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:17:30.619884 11404 tablet_service.cc:1467] Processing CreateTablet for tablet 11d032a472f147d9a7a9a83a6b6f51fc (DEFAULT_TABLE table=test-table [id=396fb4fe48bf4c9bb4b02030131b9b16]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:17:30.620777 11586 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 11d032a472f147d9a7a9a83a6b6f51fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:17:30.621192 11404 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 11d032a472f147d9a7a9a83a6b6f51fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:17:30.622308 11776 tablet_service.cc:1467] Processing CreateTablet for tablet 11d032a472f147d9a7a9a83a6b6f51fc (DEFAULT_TABLE table=test-table [id=396fb4fe48bf4c9bb4b02030131b9b16]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:17:30.623716 11776 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 11d032a472f147d9a7a9a83a6b6f51fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:17:30.632404 11899 tablet_bootstrap.cc:492] T 11d032a472f147d9a7a9a83a6b6f51fc P fd77991ca716451689a298661577abf0: Bootstrap starting.
I20260807 18:17:30.636334 11899 tablet_bootstrap.cc:654] T 11d032a472f147d9a7a9a83a6b6f51fc P fd77991ca716451689a298661577abf0: Neither blocks nor log segments found. Creating new log.
I20260807 18:17:30.637774 11899 log.cc:824] T 11d032a472f147d9a7a9a83a6b6f51fc P fd77991ca716451689a298661577abf0: Log is configured to *not* fsync() on all Append() calls
I20260807 18:17:30.638547 11900 tablet_bootstrap.cc:492] T 11d032a472f147d9a7a9a83a6b6f51fc P bf1991da7e974a75b86b39d7abef9475: Bootstrap starting.
I20260807 18:17:30.640167 11902 tablet_bootstrap.cc:492] T 11d032a472f147d9a7a9a83a6b6f51fc P 7bcf4ff6892d451faeadaa7f385f9f26: Bootstrap starting.
I20260807 18:17:30.642884 11900 tablet_bootstrap.cc:654] T 11d032a472f147d9a7a9a83a6b6f51fc P bf1991da7e974a75b86b39d7abef9475: Neither blocks nor log segments found. Creating new log.
I20260807 18:17:30.643435 11902 tablet_bootstrap.cc:654] T 11d032a472f147d9a7a9a83a6b6f51fc P 7bcf4ff6892d451faeadaa7f385f9f26: Neither blocks nor log segments found. Creating new log.
I20260807 18:17:30.644176 11899 tablet_bootstrap.cc:492] T 11d032a472f147d9a7a9a83a6b6f51fc P fd77991ca716451689a298661577abf0: No bootstrap required, opened a new log
I20260807 18:17:30.644271 11900 log.cc:824] T 11d032a472f147d9a7a9a83a6b6f51fc P bf1991da7e974a75b86b39d7abef9475: Log is configured to *not* fsync() on all Append() calls
I20260807 18:17:30.644479 11899 ts_tablet_manager.cc:1397] T 11d032a472f147d9a7a9a83a6b6f51fc P fd77991ca716451689a298661577abf0: Time spent bootstrapping tablet: real 0.012s user 0.005s sys 0.005s
I20260807 18:17:30.644636 11902 log.cc:824] T 11d032a472f147d9a7a9a83a6b6f51fc P 7bcf4ff6892d451faeadaa7f385f9f26: Log is configured to *not* fsync() on all Append() calls
I20260807 18:17:30.647650 11900 tablet_bootstrap.cc:492] T 11d032a472f147d9a7a9a83a6b6f51fc P bf1991da7e974a75b86b39d7abef9475: No bootstrap required, opened a new log
I20260807 18:17:30.647850 11902 tablet_bootstrap.cc:492] T 11d032a472f147d9a7a9a83a6b6f51fc P 7bcf4ff6892d451faeadaa7f385f9f26: No bootstrap required, opened a new log
I20260807 18:17:30.647958 11900 ts_tablet_manager.cc:1397] T 11d032a472f147d9a7a9a83a6b6f51fc P bf1991da7e974a75b86b39d7abef9475: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.003s
I20260807 18:17:30.648120 11902 ts_tablet_manager.cc:1397] T 11d032a472f147d9a7a9a83a6b6f51fc P 7bcf4ff6892d451faeadaa7f385f9f26: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260807 18:17:30.656165 11900 raft_consensus.cc:348] T 11d032a472f147d9a7a9a83a6b6f51fc P bf1991da7e974a75b86b39d7abef9475 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bcf4ff6892d451faeadaa7f385f9f26" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 36677 } } peers { permanent_uuid: "bf1991da7e974a75b86b39d7abef9475" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 39673 } } peers { permanent_uuid: "fd77991ca716451689a298661577abf0" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 35983 } }
I20260807 18:17:30.656147 11899 raft_consensus.cc:348] T 11d032a472f147d9a7a9a83a6b6f51fc P fd77991ca716451689a298661577abf0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bcf4ff6892d451faeadaa7f385f9f26" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 36677 } } peers { permanent_uuid: "bf1991da7e974a75b86b39d7abef9475" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 39673 } } peers { permanent_uuid: "fd77991ca716451689a298661577abf0" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 35983 } }
I20260807 18:17:30.656540 11900 raft_consensus.cc:374] T 11d032a472f147d9a7a9a83a6b6f51fc P bf1991da7e974a75b86b39d7abef9475 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:17:30.656592 11899 raft_consensus.cc:374] T 11d032a472f147d9a7a9a83a6b6f51fc P fd77991ca716451689a298661577abf0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:17:30.656641 11900 raft_consensus.cc:729] T 11d032a472f147d9a7a9a83a6b6f51fc P bf1991da7e974a75b86b39d7abef9475 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf1991da7e974a75b86b39d7abef9475, State: Initialized, Role: FOLLOWER
I20260807 18:17:30.656701 11899 raft_consensus.cc:729] T 11d032a472f147d9a7a9a83a6b6f51fc P fd77991ca716451689a298661577abf0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd77991ca716451689a298661577abf0, State: Initialized, Role: FOLLOWER
I20260807 18:17:30.657281 11900 consensus_queue.cc:260] T 11d032a472f147d9a7a9a83a6b6f51fc P bf1991da7e974a75b86b39d7abef9475 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bcf4ff6892d451faeadaa7f385f9f26" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 36677 } } peers { permanent_uuid: "bf1991da7e974a75b86b39d7abef9475" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 39673 } } peers { permanent_uuid: "fd77991ca716451689a298661577abf0" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 35983 } }
I20260807 18:17:30.657387 11899 consensus_queue.cc:260] T 11d032a472f147d9a7a9a83a6b6f51fc P fd77991ca716451689a298661577abf0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bcf4ff6892d451faeadaa7f385f9f26" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 36677 } } peers { permanent_uuid: "bf1991da7e974a75b86b39d7abef9475" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 39673 } } peers { permanent_uuid: "fd77991ca716451689a298661577abf0" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 35983 } }
I20260807 18:17:30.658774 11902 raft_consensus.cc:348] T 11d032a472f147d9a7a9a83a6b6f51fc P 7bcf4ff6892d451faeadaa7f385f9f26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bcf4ff6892d451faeadaa7f385f9f26" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 36677 } } peers { permanent_uuid: "bf1991da7e974a75b86b39d7abef9475" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 39673 } } peers { permanent_uuid: "fd77991ca716451689a298661577abf0" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 35983 } }
I20260807 18:17:30.659222 11902 raft_consensus.cc:374] T 11d032a472f147d9a7a9a83a6b6f51fc P 7bcf4ff6892d451faeadaa7f385f9f26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:17:30.659329 11902 raft_consensus.cc:729] T 11d032a472f147d9a7a9a83a6b6f51fc P 7bcf4ff6892d451faeadaa7f385f9f26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7bcf4ff6892d451faeadaa7f385f9f26, State: Initialized, Role: FOLLOWER
I20260807 18:17:30.659349 11899 ts_tablet_manager.cc:1428] T 11d032a472f147d9a7a9a83a6b6f51fc P fd77991ca716451689a298661577abf0: Time spent starting tablet: real 0.015s user 0.011s sys 0.003s
I20260807 18:17:30.659996 11902 consensus_queue.cc:260] T 11d032a472f147d9a7a9a83a6b6f51fc P 7bcf4ff6892d451faeadaa7f385f9f26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bcf4ff6892d451faeadaa7f385f9f26" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 36677 } } peers { permanent_uuid: "bf1991da7e974a75b86b39d7abef9475" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 39673 } } peers { permanent_uuid: "fd77991ca716451689a298661577abf0" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 35983 } }
I20260807 18:17:30.660285 11678 heartbeater.cc:503] Master 127.18.154.62:42251 was elected leader, sending a full tablet report...
I20260807 18:17:30.660904 11900 ts_tablet_manager.cc:1428] T 11d032a472f147d9a7a9a83a6b6f51fc P bf1991da7e974a75b86b39d7abef9475: Time spent starting tablet: real 0.013s user 0.012s sys 0.001s
I20260807 18:17:30.661655 11902 ts_tablet_manager.cc:1428] T 11d032a472f147d9a7a9a83a6b6f51fc P 7bcf4ff6892d451faeadaa7f385f9f26: Time spent starting tablet: real 0.013s user 0.015s sys 0.000s
I20260807 18:17:30.661813 11871 heartbeater.cc:503] Master 127.18.154.62:42251 was elected leader, sending a full tablet report...
W20260807 18:17:30.711863 11679 tablet.cc:2369] T 11d032a472f147d9a7a9a83a6b6f51fc P bf1991da7e974a75b86b39d7abef9475: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 18:17:30.767244 11872 tablet.cc:2369] T 11d032a472f147d9a7a9a83a6b6f51fc P 7bcf4ff6892d451faeadaa7f385f9f26: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 18:17:30.898614 11492 tablet.cc:2369] T 11d032a472f147d9a7a9a83a6b6f51fc P fd77991ca716451689a298661577abf0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:17:30.924367 11911 raft_consensus.cc:482] T 11d032a472f147d9a7a9a83a6b6f51fc P fd77991ca716451689a298661577abf0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:17:30.924719 11911 raft_consensus.cc:504] T 11d032a472f147d9a7a9a83a6b6f51fc P fd77991ca716451689a298661577abf0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bcf4ff6892d451faeadaa7f385f9f26" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 36677 } } peers { permanent_uuid: "bf1991da7e974a75b86b39d7abef9475" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 39673 } } peers { permanent_uuid: "fd77991ca716451689a298661577abf0" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 35983 } }
I20260807 18:17:30.926578 11911 leader_election.cc:288] T 11d032a472f147d9a7a9a83a6b6f51fc P fd77991ca716451689a298661577abf0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7bcf4ff6892d451faeadaa7f385f9f26 (127.18.154.3:36677), bf1991da7e974a75b86b39d7abef9475 (127.18.154.2:39673)
I20260807 18:17:30.934962 11892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:30.927123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.1:58503 (local address 127.18.154.3:36677)
0807 18:17:30.927376 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:30.927382 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:30.927620 (+ 238us) server_negotiation.cc:407] Connection header received
0807 18:17:30.927904 (+ 284us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:30.927914 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:30.928026 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:30.928134 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:17:30.928879 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:30.929974 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:30.932506 (+ 2532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:30.933806 (+ 1300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:30.933972 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:30.934062 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:30.934556 (+ 494us) server_negotiation.cc:299] Negotiation successful
0807 18:17:30.934689 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260807 18:17:30.934963 11898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:30.927906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.1:57481 (local address 127.18.154.2:39673)
0807 18:17:30.928119 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:30.928128 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:30.928606 (+ 478us) server_negotiation.cc:407] Connection header received
0807 18:17:30.928810 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:30.928817 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:30.928901 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:30.929006 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:17:30.929469 (+ 463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:30.930382 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:30.932684 (+ 2302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:30.933941 (+ 1257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:30.934079 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:30.934143 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:30.934556 (+ 413us) server_negotiation.cc:299] Negotiation successful
0807 18:17:30.934689 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 18:17:30.935199 11913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:30.926866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.3:36677 (local address 127.18.154.1:58503)
0807 18:17:30.927493 (+ 627us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:30.927510 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 18:17:30.927691 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:17:30.928243 (+ 552us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:17:30.928248 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:17:30.928284 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:17:30.928730 (+ 446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:30.928746 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:30.930097 (+ 1351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:30.930100 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:30.932342 (+ 2242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:30.932357 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:30.934043 (+ 1686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:30.934051 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:30.934332 (+ 281us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:30.934414 (+ 82us) client_negotiation.cc:769] Sending connection context
0807 18:17:30.934515 (+ 101us) client_negotiation.cc:240] Negotiation successful
0807 18:17:30.934718 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":515,"mutex_wait_us":49,"thread_start_us":340,"threads_started":1}
I20260807 18:17:30.935698 11914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:30.927641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.2:39673 (local address 127.18.154.1:57481)
0807 18:17:30.928500 (+ 859us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:30.928517 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 18:17:30.928654 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:17:30.929023 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:17:30.929026 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:17:30.929046 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:17:30.929331 (+ 285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:30.929338 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:30.930511 (+ 1173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:30.930516 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:30.932551 (+ 2035us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:30.932561 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:30.934155 (+ 1594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:30.934161 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:30.934411 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:30.934437 (+ 26us) client_negotiation.cc:769] Sending connection context
0807 18:17:30.934515 (+ 78us) client_negotiation.cc:240] Negotiation successful
0807 18:17:30.934721 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":720,"mutex_wait_us":94,"thread_start_us":488,"threads_started":1}
I20260807 18:17:30.936218 11804 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11d032a472f147d9a7a9a83a6b6f51fc" candidate_uuid: "fd77991ca716451689a298661577abf0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bcf4ff6892d451faeadaa7f385f9f26" is_pre_election: true
I20260807 18:17:30.936692 11804 raft_consensus.cc:2454] T 11d032a472f147d9a7a9a83a6b6f51fc P 7bcf4ff6892d451faeadaa7f385f9f26 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fd77991ca716451689a298661577abf0 in term 0.
I20260807 18:17:30.937387 11347 leader_election.cc:302] T 11d032a472f147d9a7a9a83a6b6f51fc P fd77991ca716451689a298661577abf0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bcf4ff6892d451faeadaa7f385f9f26, fd77991ca716451689a298661577abf0; no voters:
I20260807 18:17:30.938046 11911 raft_consensus.cc:2781] T 11d032a472f147d9a7a9a83a6b6f51fc P fd77991ca716451689a298661577abf0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:17:30.938277 11911 raft_consensus.cc:482] T 11d032a472f147d9a7a9a83a6b6f51fc P fd77991ca716451689a298661577abf0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:17:30.938081 11610 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11d032a472f147d9a7a9a83a6b6f51fc" candidate_uuid: "fd77991ca716451689a298661577abf0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf1991da7e974a75b86b39d7abef9475" is_pre_election: true
I20260807 18:17:30.938364 11911 raft_consensus.cc:3037] T 11d032a472f147d9a7a9a83a6b6f51fc P fd77991ca716451689a298661577abf0 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:17:30.938568 11610 raft_consensus.cc:2454] T 11d032a472f147d9a7a9a83a6b6f51fc P bf1991da7e974a75b86b39d7abef9475 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fd77991ca716451689a298661577abf0 in term 0.
I20260807 18:17:30.940661 11911 raft_consensus.cc:504] T 11d032a472f147d9a7a9a83a6b6f51fc P fd77991ca716451689a298661577abf0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bcf4ff6892d451faeadaa7f385f9f26" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 36677 } } peers { permanent_uuid: "bf1991da7e974a75b86b39d7abef9475" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 39673 } } peers { permanent_uuid: "fd77991ca716451689a298661577abf0" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 35983 } }
I20260807 18:17:30.941243 11911 leader_election.cc:288] T 11d032a472f147d9a7a9a83a6b6f51fc P fd77991ca716451689a298661577abf0 [CANDIDATE]: Term 1 election: Requested vote from peers 7bcf4ff6892d451faeadaa7f385f9f26 (127.18.154.3:36677), bf1991da7e974a75b86b39d7abef9475 (127.18.154.2:39673)
I20260807 18:17:30.941717 11804 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11d032a472f147d9a7a9a83a6b6f51fc" candidate_uuid: "fd77991ca716451689a298661577abf0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bcf4ff6892d451faeadaa7f385f9f26"
I20260807 18:17:30.941753 11610 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11d032a472f147d9a7a9a83a6b6f51fc" candidate_uuid: "fd77991ca716451689a298661577abf0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf1991da7e974a75b86b39d7abef9475"
I20260807 18:17:30.941926 11804 raft_consensus.cc:3037] T 11d032a472f147d9a7a9a83a6b6f51fc P 7bcf4ff6892d451faeadaa7f385f9f26 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:17:30.941946 11610 raft_consensus.cc:3037] T 11d032a472f147d9a7a9a83a6b6f51fc P bf1991da7e974a75b86b39d7abef9475 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:17:30.944099 11610 raft_consensus.cc:2454] T 11d032a472f147d9a7a9a83a6b6f51fc P bf1991da7e974a75b86b39d7abef9475 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fd77991ca716451689a298661577abf0 in term 1.
I20260807 18:17:30.944124 11804 raft_consensus.cc:2454] T 11d032a472f147d9a7a9a83a6b6f51fc P 7bcf4ff6892d451faeadaa7f385f9f26 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fd77991ca716451689a298661577abf0 in term 1.
I20260807 18:17:30.944520 11347 leader_election.cc:302] T 11d032a472f147d9a7a9a83a6b6f51fc P fd77991ca716451689a298661577abf0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bf1991da7e974a75b86b39d7abef9475, fd77991ca716451689a298661577abf0; no voters:
I20260807 18:17:30.944802 11911 raft_consensus.cc:2781] T 11d032a472f147d9a7a9a83a6b6f51fc P fd77991ca716451689a298661577abf0 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:17:30.945318 11911 raft_consensus.cc:686] T 11d032a472f147d9a7a9a83a6b6f51fc P fd77991ca716451689a298661577abf0 [term 1 LEADER]: Becoming Leader. State: Replica: fd77991ca716451689a298661577abf0, State: Running, Role: LEADER
I20260807 18:17:30.945916 11911 consensus_queue.cc:237] T 11d032a472f147d9a7a9a83a6b6f51fc P fd77991ca716451689a298661577abf0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bcf4ff6892d451faeadaa7f385f9f26" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 36677 } } peers { permanent_uuid: "bf1991da7e974a75b86b39d7abef9475" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 39673 } } peers { permanent_uuid: "fd77991ca716451689a298661577abf0" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 35983 } }
I20260807 18:17:30.950774 11227 catalog_manager.cc:5495] T 11d032a472f147d9a7a9a83a6b6f51fc P fd77991ca716451689a298661577abf0 reported cstate change: term changed from 0 to 1, leader changed from <none> to fd77991ca716451689a298661577abf0 (127.18.154.1). New cstate: current_term: 1 leader_uuid: "fd77991ca716451689a298661577abf0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bcf4ff6892d451faeadaa7f385f9f26" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 36677 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bf1991da7e974a75b86b39d7abef9475" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 39673 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fd77991ca716451689a298661577abf0" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 35983 } health_report { overall_health: HEALTHY } } }
I20260807 18:17:30.982718 11895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:30.977385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60854 (local address 127.18.154.1:35983)
0807 18:17:30.977631 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:30.977636 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:30.977655 (+ 19us) server_negotiation.cc:407] Connection header received
0807 18:17:30.977749 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:30.977753 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:30.977830 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:30.977955 (+ 125us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:17:30.978323 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:30.979070 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:30.980495 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:30.980956 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:30.981049 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:30.981513 (+ 464us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:17:30.982117 (+ 604us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:17:30.982255 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:30.982354 (+ 99us) server_negotiation.cc:299] Negotiation successful
0807 18:17:30.982421 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 18:17:30.991360 11610 raft_consensus.cc:1261] T 11d032a472f147d9a7a9a83a6b6f51fc P bf1991da7e974a75b86b39d7abef9475 [term 1 FOLLOWER]: Refusing update from remote peer fd77991ca716451689a298661577abf0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:17:30.991456 11804 raft_consensus.cc:1261] T 11d032a472f147d9a7a9a83a6b6f51fc P 7bcf4ff6892d451faeadaa7f385f9f26 [term 1 FOLLOWER]: Refusing update from remote peer fd77991ca716451689a298661577abf0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:17:30.992441 11915 consensus_queue.cc:1035] T 11d032a472f147d9a7a9a83a6b6f51fc P fd77991ca716451689a298661577abf0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bf1991da7e974a75b86b39d7abef9475" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 39673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:17:30.992830 11911 consensus_queue.cc:1035] T 11d032a472f147d9a7a9a83a6b6f51fc P fd77991ca716451689a298661577abf0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bcf4ff6892d451faeadaa7f385f9f26" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 36677 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:17:31.012609 11921 mvcc.cc:205] Tried to move back new op lower bound from 7315974762450432000 to 7315974762278313984. Current Snapshot: MvccSnapshot[applied={T|T < 7315974762450432000 or (T in {7315974762450432000})}]
I20260807 18:17:31.019930 11226 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40320:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:17:31.020241 11226 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40320:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:17:31.023911 11226 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 578da878099941b09adbc32a1481f61e: Sending DeleteTablet for 3 replicas of tablet 11d032a472f147d9a7a9a83a6b6f51fc
I20260807 18:17:31.025156 11404 tablet_service.cc:1514] Processing DeleteTablet for tablet 11d032a472f147d9a7a9a83a6b6f51fc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:17:31 UTC) from {username='kudu', principal='kudu/127.18.154.62@KRBTEST.COM'} at 127.0.0.1:60844
I20260807 18:17:31.025631 11776 tablet_service.cc:1514] Processing DeleteTablet for tablet 11d032a472f147d9a7a9a83a6b6f51fc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:17:31 UTC) from {username='kudu', principal='kudu/127.18.154.62@KRBTEST.COM'} at 127.0.0.1:33264
I20260807 18:17:31.026355 11586 tablet_service.cc:1514] Processing DeleteTablet for tablet 11d032a472f147d9a7a9a83a6b6f51fc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:17:31 UTC) from {username='kudu', principal='kudu/127.18.154.62@KRBTEST.COM'} at 127.0.0.1:54374
I20260807 18:17:31.026726 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 11313
I20260807 18:17:31.040042 11938 tablet_replica.cc:330] T 11d032a472f147d9a7a9a83a6b6f51fc P bf1991da7e974a75b86b39d7abef9475: stopping tablet replica
I20260807 18:17:31.040699 11938 raft_consensus.cc:2229] T 11d032a472f147d9a7a9a83a6b6f51fc P bf1991da7e974a75b86b39d7abef9475 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:17:31.041424 11938 raft_consensus.cc:2258] T 11d032a472f147d9a7a9a83a6b6f51fc P bf1991da7e974a75b86b39d7abef9475 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:17:31.043700 11937 tablet_replica.cc:330] T 11d032a472f147d9a7a9a83a6b6f51fc P 7bcf4ff6892d451faeadaa7f385f9f26: stopping tablet replica
I20260807 18:17:31.044305 11937 raft_consensus.cc:2229] T 11d032a472f147d9a7a9a83a6b6f51fc P 7bcf4ff6892d451faeadaa7f385f9f26 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:17:31.044898 11937 raft_consensus.cc:2258] T 11d032a472f147d9a7a9a83a6b6f51fc P 7bcf4ff6892d451faeadaa7f385f9f26 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:17:31.045859 11935 tablet_replica.cc:330] T 11d032a472f147d9a7a9a83a6b6f51fc P fd77991ca716451689a298661577abf0: stopping tablet replica
I20260807 18:17:31.046351 11935 raft_consensus.cc:2229] T 11d032a472f147d9a7a9a83a6b6f51fc P fd77991ca716451689a298661577abf0 [term 1 LEADER]: Raft consensus shutting down.
I20260807 18:17:31.046691 11938 ts_tablet_manager.cc:1905] T 11d032a472f147d9a7a9a83a6b6f51fc P bf1991da7e974a75b86b39d7abef9475: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:17:31.047115 11937 ts_tablet_manager.cc:1905] T 11d032a472f147d9a7a9a83a6b6f51fc P 7bcf4ff6892d451faeadaa7f385f9f26: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:17:31.053452 11937 ts_tablet_manager.cc:1918] T 11d032a472f147d9a7a9a83a6b6f51fc P 7bcf4ff6892d451faeadaa7f385f9f26: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:17:31.053658 11937 log.cc:1196] T 11d032a472f147d9a7a9a83a6b6f51fc P 7bcf4ff6892d451faeadaa7f385f9f26: Deleting WAL directory at /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126328808105-19048-0/minicluster-data/ts-2/wal/wals/11d032a472f147d9a7a9a83a6b6f51fc
I20260807 18:17:31.053985 11938 ts_tablet_manager.cc:1918] T 11d032a472f147d9a7a9a83a6b6f51fc P bf1991da7e974a75b86b39d7abef9475: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:17:31.054150 11938 log.cc:1196] T 11d032a472f147d9a7a9a83a6b6f51fc P bf1991da7e974a75b86b39d7abef9475: Deleting WAL directory at /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786126328808105-19048-0/minicluster-data/ts-1/wal/wals/11d032a472f147d9a7a9a83a6b6f51fc
I20260807 18:17:31.054169 11935 raft_consensus.cc:2258] T 11d032a472f147d9a7a9a83a6b6f51fc P fd77991ca716451689a298661577abf0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:17:31.054265 11937 ts_tablet_manager.cc:1939] T 11d032a472f147d9a7a9a83a6b6f51fc P 7bcf4ff6892d451faeadaa7f385f9f26: Deleting consensus metadata
I20260807 18:17:31.054785 11938 ts_tablet_manager.cc:1939] T 11d032a472f147d9a7a9a83a6b6f51fc P bf1991da7e974a75b86b39d7abef9475: Deleting consensus metadata
Aug 07 18:17:31 dist-test-slave-gkw7 krb5kdc[11159](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126648, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.154.1@KRBTEST.COM
I20260807 18:17:31.056538 11204 catalog_manager.cc:4841] TS 7bcf4ff6892d451faeadaa7f385f9f26 (127.18.154.3:36677): tablet 11d032a472f147d9a7a9a83a6b6f51fc (table test-table [id=396fb4fe48bf4c9bb4b02030131b9b16]) successfully deleted
I20260807 18:17:31.056895 11204 catalog_manager.cc:4841] TS bf1991da7e974a75b86b39d7abef9475 (127.18.154.2:39673): tablet 11d032a472f147d9a7a9a83a6b6f51fc (table test-table [id=396fb4fe48bf4c9bb4b02030131b9b16]) successfully deleted
I20260807 18:17:31.058804 11935 ts_tablet_manager.cc:1905] T 11d032a472f147d9a7a9a83a6b6f51fc P fd77991ca716451689a298661577abf0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:17:31.063165 11895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:31.028773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60862 (local address 127.18.154.1:35983)
0807 18:17:31.029002 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:31.029015 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:31.041604 (+ 12589us) server_negotiation.cc:407] Connection header received
0807 18:17:31.042562 (+ 958us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:31.042571 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:31.042694 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:31.042838 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:17:31.045652 (+ 2814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:31.046886 (+ 1234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:31.048834 (+ 1948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:31.049510 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:31.049632 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:31.056513 (+ 6881us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:17:31.056529 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:17:31.056550 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:17:31.056602 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:17:31.060048 (+ 3446us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:17:31.060783 (+ 735us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:17:31.060790 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:17:31.060793 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:17:31.060956 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:17:31.061410 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:17:31.061417 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:17:31.061420 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:17:31.061859 (+ 439us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:17:31.062178 (+ 319us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:31.062519 (+ 341us) server_negotiation.cc:299] Negotiation successful
0807 18:17:31.062792 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260807 18:17:31.063907 11482 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:17:31.165757 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 11313
I20260807 18:17:31.193255 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 11499
W20260807 18:17:31.193202 11198 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.154.1:35983 (error 108)
I20260807 18:17:31.195287 11943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:31.194496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.1:35983 (local address 127.0.0.1:60868)
0807 18:17:31.194902 (+ 406us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:31.195061 (+ 159us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.1:35983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":308,"thread_start_us":215,"threads_started":1}
W20260807 18:17:31.195585 11198 catalog_manager.cc:4584] TS fd77991ca716451689a298661577abf0 (127.18.154.1:35983): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 11d032a472f147d9a7a9a83a6b6f51fc: Network error: Client connection negotiation failed: client connection to 127.18.154.1:35983: connect: Connection refused (error 111)
Aug 07 18:17:31 dist-test-slave-gkw7 krb5kdc[11159](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126648, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.154.2@KRBTEST.COM
I20260807 18:17:31.209712 11898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:31.194147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54380 (local address 127.18.154.2:39673)
0807 18:17:31.194330 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:31.194336 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:31.194468 (+ 132us) server_negotiation.cc:407] Connection header received
0807 18:17:31.194747 (+ 279us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:31.194754 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:31.194853 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:31.194949 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:17:31.196484 (+ 1535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:31.197226 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:31.198762 (+ 1536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:31.199237 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:31.199333 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:31.203873 (+ 4540us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:17:31.203891 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:17:31.203908 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:17:31.203948 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:17:31.206882 (+ 2934us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:17:31.207599 (+ 717us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:17:31.207606 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:17:31.207609 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:17:31.207742 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:17:31.208080 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:17:31.208083 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:17:31.208085 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:17:31.208505 (+ 420us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:17:31.208744 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:31.208991 (+ 247us) server_negotiation.cc:299] Negotiation successful
0807 18:17:31.209171 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260807 18:17:31.210407 11669 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:17:31.249367 11943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:31.248743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.1:35983 (local address 127.0.0.1:60876)
0807 18:17:31.249052 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:31.249182 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.1:35983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137}
I20260807 18:17:31.250644 11943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:31.250205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.1:35983 (local address 127.0.0.1:60884)
0807 18:17:31.250388 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:31.250476 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.1:35983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260807 18:17:31.304804 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 11499
I20260807 18:17:31.311537 11943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:31.310898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.1:35983 (local address 127.0.0.1:60900)
0807 18:17:31.311170 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:31.311314 (+ 144us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.1:35983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260807 18:17:31.313001 11943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:31.312490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.1:35983 (local address 127.0.0.1:60916)
0807 18:17:31.312697 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:31.312798 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.1:35983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260807 18:17:31.330360 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 11687
Aug 07 18:17:31 dist-test-slave-gkw7 krb5kdc[11159](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126648, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.154.3@KRBTEST.COM
I20260807 18:17:31.353261 11892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:31.331250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33276 (local address 127.18.154.3:36677)
0807 18:17:31.331442 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:31.331447 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:31.331468 (+ 21us) server_negotiation.cc:407] Connection header received
0807 18:17:31.331676 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:31.331681 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:31.331752 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:31.331852 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:17:31.333322 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:31.334325 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:31.335769 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:31.336258 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:31.336381 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:31.346804 (+ 10423us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:17:31.346823 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:17:31.346841 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:17:31.346884 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:17:31.350117 (+ 3233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:17:31.350995 (+ 878us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:17:31.351003 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:17:31.351006 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:17:31.351213 (+ 207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:17:31.351616 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:17:31.351620 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:17:31.351622 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:17:31.352051 (+ 429us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:17:31.352340 (+ 289us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:31.352758 (+ 418us) server_negotiation.cc:299] Negotiation successful
0807 18:17:31.352959 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260807 18:17:31.354005 11866 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:17:31.393806 11943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:31.393243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.1:35983 (local address 127.0.0.1:60930)
0807 18:17:31.393473 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:31.393620 (+ 147us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.1:35983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260807 18:17:31.394979 11943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:31.394542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.1:35983 (local address 127.0.0.1:60938)
0807 18:17:31.394755 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:31.394826 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.1:35983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97}
I20260807 18:17:31.444751 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 11687
I20260807 18:17:31.465732 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 11174
I20260807 18:17:31.466943 11270 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:17:31.580755 11943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:31.580107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.1:35983 (local address 127.0.0.1:60954)
0807 18:17:31.580447 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:31.580558 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.1:35983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":186}
I20260807 18:17:31.580801 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 11174
2026-08-07T18:17:31Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (4953 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-07T18:17:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:17:31Z Disabled control of system clock
I20260807 18:17:31.680541 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126328808105-19048-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126328808105-19048-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126328808105-19048-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.154.62:45053
--webserver_interface=127.18.154.62
--webserver_port=0
--builtin_ntp_servers=127.18.154.20:34085
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.154.62:45053
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 18:17:32.036798 11958 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:17:32.037120 11958 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:17:32.037257 11958 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:17:32.045017 11958 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:17:32.045154 11958 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:17:32.045230 11958 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:17:32.045325 11958 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:17:32.045414 11958 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:17:32.055289 11958 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.154.20:34085
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126328808105-19048-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126328808105-19048-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.154.62:45053
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.154.62:45053
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.154.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126328808105-19048-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:17:32.056711 11958 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:17:32.058674 11958 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:17:32.071599 11965 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:17:32.071645 11970 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:17:32.071684 11966 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:17:32.072975 11958 server_base.cc:1034] running on GCE node
I20260807 18:17:32.073968 11958 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:17:32.075574 11958 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:17:32.076817 11958 hybrid_clock.cc:648] HybridClock initialized: now 1786126652076760 us; error 48 us; skew 500 ppm
I20260807 18:17:32.077212 11958 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 18:17:32.079793 11958 webserver.cc:460] Webserver started at http://127.18.154.62:34991/ using document root <none> and password file <none>
I20260807 18:17:32.080608 11958 fs_manager.cc:359] Metadata directory not provided
I20260807 18:17:32.080703 11958 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:17:32.081055 11958 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:17:32.084311 11958 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126328808105-19048-0/minicluster-data/master-0/data/instance:
uuid: "79210462dc754603b9148cfb6d4426ba"
format_stamp: "Formatted at 2026-08-07 18:17:32 on dist-test-slave-gkw7"
server_key: "2fdf8dec4901f715c5031e88b6cf6320"
server_key_iv: "1b230d20ff09840230386a66f46b99bd"
server_key_version: "encryptionkey@0"
I20260807 18:17:32.085289 11958 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126328808105-19048-0/minicluster-data/master-0/wal/instance:
uuid: "79210462dc754603b9148cfb6d4426ba"
format_stamp: "Formatted at 2026-08-07 18:17:32 on dist-test-slave-gkw7"
server_key: "2fdf8dec4901f715c5031e88b6cf6320"
server_key_iv: "1b230d20ff09840230386a66f46b99bd"
server_key_version: "encryptionkey@0"
I20260807 18:17:32.091077 11958 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260807 18:17:32.094985 11977 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:17:32.096545 11958 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 18:17:32.096760 11958 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126328808105-19048-0/minicluster-data/master-0/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126328808105-19048-0/minicluster-data/master-0/wal
uuid: "79210462dc754603b9148cfb6d4426ba"
format_stamp: "Formatted at 2026-08-07 18:17:32 on dist-test-slave-gkw7"
server_key: "2fdf8dec4901f715c5031e88b6cf6320"
server_key_iv: "1b230d20ff09840230386a66f46b99bd"
server_key_version: "encryptionkey@0"
I20260807 18:17:32.096994 11958 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126328808105-19048-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126328808105-19048-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126328808105-19048-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:17:32.122244 11958 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:17:32.123195 11958 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:17:32.123512 11958 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:17:32.143749 11958 rpc_server.cc:307] RPC server started. Bound to: 127.18.154.62:45053
I20260807 18:17:32.143810 12047 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.154.62:45053 every 8 connection(s)
I20260807 18:17:32.145298 11958 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
I20260807 18:17:32.149857 12048 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:17:32.151857 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 11958
I20260807 18:17:32.152051 19048 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126328808105-19048-0/minicluster-data/master-0/wal/instance
I20260807 18:17:32.152647 19048 external_mini_cluster.cc:1428] Setting key 05f5a7c6632bdd3fef2934a29ce5490a
I20260807 18:17:32.166172 12048 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 79210462dc754603b9148cfb6d4426ba: Bootstrap starting.
I20260807 18:17:32.168329 12051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:32.155177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57532 (local address 127.18.154.62:45053)
0807 18:17:32.156540 (+ 1363us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:32.156552 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:32.156597 (+ 45us) server_negotiation.cc:407] Connection header received
0807 18:17:32.158282 (+ 1685us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:32.158308 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:32.158827 (+ 519us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:32.159166 (+ 339us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:17:32.159847 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:32.161167 (+ 1320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:32.163193 (+ 2026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:32.163923 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:32.164055 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:32.165453 (+ 1398us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:17:32.165462 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:17:32.165478 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:17:32.166220 (+ 742us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:17:32.166344 (+ 124us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:17:32.166380 (+ 36us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:17:32.166504 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:32.166827 (+ 323us) server_negotiation.cc:299] Negotiation successful
0807 18:17:32.167220 (+ 393us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":785,"thread_start_us":439,"threads_started":1}
I20260807 18:17:32.169814 12048 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 79210462dc754603b9148cfb6d4426ba: Neither blocks nor log segments found. Creating new log.
I20260807 18:17:32.171204 12048 log.cc:824] T 00000000000000000000000000000000 P 79210462dc754603b9148cfb6d4426ba: Log is configured to *not* fsync() on all Append() calls
I20260807 18:17:32.174098 12048 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 79210462dc754603b9148cfb6d4426ba: No bootstrap required, opened a new log
I20260807 18:17:32.182123 12048 raft_consensus.cc:348] T 00000000000000000000000000000000 P 79210462dc754603b9148cfb6d4426ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79210462dc754603b9148cfb6d4426ba" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 45053 } }
I20260807 18:17:32.182600 12048 raft_consensus.cc:374] T 00000000000000000000000000000000 P 79210462dc754603b9148cfb6d4426ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:17:32.182715 12048 raft_consensus.cc:729] T 00000000000000000000000000000000 P 79210462dc754603b9148cfb6d4426ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79210462dc754603b9148cfb6d4426ba, State: Initialized, Role: FOLLOWER
I20260807 18:17:32.183388 12048 consensus_queue.cc:260] T 00000000000000000000000000000000 P 79210462dc754603b9148cfb6d4426ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79210462dc754603b9148cfb6d4426ba" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 45053 } }
I20260807 18:17:32.183637 12048 raft_consensus.cc:388] T 00000000000000000000000000000000 P 79210462dc754603b9148cfb6d4426ba [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:17:32.183775 12048 raft_consensus.cc:482] T 00000000000000000000000000000000 P 79210462dc754603b9148cfb6d4426ba [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:17:32.183956 12048 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 79210462dc754603b9148cfb6d4426ba [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:17:32.186630 12048 raft_consensus.cc:504] T 00000000000000000000000000000000 P 79210462dc754603b9148cfb6d4426ba [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79210462dc754603b9148cfb6d4426ba" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 45053 } }
I20260807 18:17:32.187117 12048 leader_election.cc:302] T 00000000000000000000000000000000 P 79210462dc754603b9148cfb6d4426ba [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 79210462dc754603b9148cfb6d4426ba; no voters:
I20260807 18:17:32.187737 12048 leader_election.cc:288] T 00000000000000000000000000000000 P 79210462dc754603b9148cfb6d4426ba [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:17:32.187987 12057 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 79210462dc754603b9148cfb6d4426ba [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:17:32.188915 12057 raft_consensus.cc:686] T 00000000000000000000000000000000 P 79210462dc754603b9148cfb6d4426ba [term 1 LEADER]: Becoming Leader. State: Replica: 79210462dc754603b9148cfb6d4426ba, State: Running, Role: LEADER
I20260807 18:17:32.189621 12057 consensus_queue.cc:237] T 00000000000000000000000000000000 P 79210462dc754603b9148cfb6d4426ba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79210462dc754603b9148cfb6d4426ba" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 45053 } }
I20260807 18:17:32.190099 12048 sys_catalog.cc:564] T 00000000000000000000000000000000 P 79210462dc754603b9148cfb6d4426ba [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:17:32.193200 12060 sys_catalog.cc:455] T 00000000000000000000000000000000 P 79210462dc754603b9148cfb6d4426ba [sys.catalog]: SysCatalogTable state changed. Reason: New leader 79210462dc754603b9148cfb6d4426ba. Latest consensus state: current_term: 1 leader_uuid: "79210462dc754603b9148cfb6d4426ba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79210462dc754603b9148cfb6d4426ba" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 45053 } } }
I20260807 18:17:32.193315 12058 sys_catalog.cc:455] T 00000000000000000000000000000000 P 79210462dc754603b9148cfb6d4426ba [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "79210462dc754603b9148cfb6d4426ba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79210462dc754603b9148cfb6d4426ba" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 45053 } } }
I20260807 18:17:32.193753 12060 sys_catalog.cc:458] T 00000000000000000000000000000000 P 79210462dc754603b9148cfb6d4426ba [sys.catalog]: This master's current role is: LEADER
I20260807 18:17:32.193799 12058 sys_catalog.cc:458] T 00000000000000000000000000000000 P 79210462dc754603b9148cfb6d4426ba [sys.catalog]: This master's current role is: LEADER
I20260807 18:17:32.194737 12068 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:17:32.202334 12068 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:17:32.213004 12068 catalog_manager.cc:1347] Generated new cluster ID: 6b93fcbf5fec41f6b447d5ed6133fff1
I20260807 18:17:32.213168 12068 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:17:32.254611 12068 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:17:32.256014 12068 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:17:32.265179 12068 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 79210462dc754603b9148cfb6d4426ba: Generated new TSK 0
I20260807 18:17:32.266295 12068 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:17:32.276410 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126328808105-19048-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126328808105-19048-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126328808105-19048-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126328808105-19048-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.154.1:0
--local_ip_for_outbound_sockets=127.18.154.1
--webserver_interface=127.18.154.1
--webserver_port=0
--tserver_master_addrs=127.18.154.62:45053
--builtin_ntp_servers=127.18.154.20:34085
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 18:17:32.632303 12085 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:17:32.632608 12085 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:17:32.632778 12085 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:17:32.640182 12085 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:17:32.640292 12085 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:17:32.640466 12085 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.154.1
I20260807 18:17:32.650076 12085 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.154.20:34085
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126328808105-19048-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126328808105-19048-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.154.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126328808105-19048-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.154.1
--webserver_port=0
--tserver_master_addrs=127.18.154.62:45053
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.154.1
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126328808105-19048-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:17:32.651659 12085 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:17:32.653497 12085 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:17:32.667218 12095 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:17:32.667258 12093 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:17:32.667896 12092 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:17:32.671145 12085 server_base.cc:1034] running on GCE node
I20260807 18:17:32.671828 12085 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:17:32.672946 12085 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:17:32.674145 12085 hybrid_clock.cc:648] HybridClock initialized: now 1786126652674079 us; error 68 us; skew 500 ppm
I20260807 18:17:32.674480 12085 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 18:17:32.676844 12085 webserver.cc:460] Webserver started at http://127.18.154.1:41685/ using document root <none> and password file <none>
I20260807 18:17:32.677634 12085 fs_manager.cc:359] Metadata directory not provided
I20260807 18:17:32.677754 12085 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:17:32.678102 12085 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:17:32.681108 12085 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126328808105-19048-0/minicluster-data/ts-0/data/instance:
uuid: "95b31a70c3184bccad36988f7c4749ff"
format_stamp: "Formatted at 2026-08-07 18:17:32 on dist-test-slave-gkw7"
server_key: "c2da6d89f44167420cda3bdaafb55003"
server_key_iv: "1e2143309228913cba6dfa3074ec9346"
server_key_version: "encryptionkey@0"
I20260807 18:17:32.681981 12085 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126328808105-19048-0/minicluster-data/ts-0/wal/instance:
uuid: "95b31a70c3184bccad36988f7c4749ff"
format_stamp: "Formatted at 2026-08-07 18:17:32 on dist-test-slave-gkw7"
server_key: "c2da6d89f44167420cda3bdaafb55003"
server_key_iv: "1e2143309228913cba6dfa3074ec9346"
server_key_version: "encryptionkey@0"
I20260807 18:17:32.687326 12085 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260807 18:17:32.691113 12107 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:17:32.692618 12085 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 18:17:32.692832 12085 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126328808105-19048-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126328808105-19048-0/minicluster-data/ts-0/wal
uuid: "95b31a70c3184bccad36988f7c4749ff"
format_stamp: "Formatted at 2026-08-07 18:17:32 on dist-test-slave-gkw7"
server_key: "c2da6d89f44167420cda3bdaafb55003"
server_key_iv: "1e2143309228913cba6dfa3074ec9346"
server_key_version: "encryptionkey@0"
I20260807 18:17:32.693132 12085 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126328808105-19048-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126328808105-19048-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126328808105-19048-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:17:32.705468 12085 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:17:32.706486 12085 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:17:32.706813 12085 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:17:32.707825 12085 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:17:32.709456 12085 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:17:32.709610 12085 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:17:32.709748 12085 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:17:32.709844 12085 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:17:32.742609 12085 rpc_server.cc:307] RPC server started. Bound to: 127.18.154.1:39589
I20260807 18:17:32.742657 12271 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.154.1:39589 every 8 connection(s)
I20260807 18:17:32.744017 12085 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126328808105-19048-0/minicluster-data/ts-0/data/info.pb
I20260807 18:17:32.749825 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 12085
I20260807 18:17:32.750023 19048 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126328808105-19048-0/minicluster-data/ts-0/wal/instance
I20260807 18:17:32.750474 19048 external_mini_cluster.cc:1428] Setting key e8f047a3de6b4d6826f011f0859f7a29
I20260807 18:17:32.753291 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126328808105-19048-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126328808105-19048-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126328808105-19048-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126328808105-19048-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.154.2:0
--local_ip_for_outbound_sockets=127.18.154.2
--webserver_interface=127.18.154.2
--webserver_port=0
--tserver_master_addrs=127.18.154.62:45053
--builtin_ntp_servers=127.18.154.20:34085
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 18:17:32.757397 12275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:32.746843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.1:56629 (local address 127.18.154.62:45053)
0807 18:17:32.747338 (+ 495us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:32.747343 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:32.748516 (+ 1173us) server_negotiation.cc:407] Connection header received
0807 18:17:32.750519 (+ 2003us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:32.750524 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:32.750621 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:32.750753 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:17:32.751931 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:32.752768 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:32.754520 (+ 1752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:32.754937 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:32.755080 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:32.756522 (+ 1442us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:17:32.756527 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:17:32.756531 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:17:32.756550 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:17:32.756591 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:17:32.756604 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:17:32.756696 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:32.756932 (+ 236us) server_negotiation.cc:299] Negotiation successful
0807 18:17:32.757040 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":383,"thread_start_us":278,"threads_started":1}
I20260807 18:17:32.758522 12277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:32.747373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:45053 (local address 127.18.154.1:56629)
0807 18:17:32.748328 (+ 955us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:32.748367 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 18:17:32.750233 (+ 1866us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:17:32.751002 (+ 769us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:17:32.751011 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:17:32.751110 (+ 99us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:17:32.751747 (+ 637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:32.751765 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:32.752922 (+ 1157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:32.752933 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:32.754367 (+ 1434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:32.754381 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:32.755183 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:32.755193 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:32.755439 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:32.756173 (+ 734us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 18:17:32.756176 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:17:32.756226 (+ 50us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 18:17:32.756311 (+ 85us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:17:32.756713 (+ 402us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:17:32.756731 (+ 18us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:17:32.756739 (+ 8us) client_negotiation.cc:769] Sending connection context
0807 18:17:32.756925 (+ 186us) client_negotiation.cc:240] Negotiation successful
0807 18:17:32.757371 (+ 446us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":303,"threads_started":1}
I20260807 18:17:32.760193 12272 heartbeater.cc:347] Connected to a master server at 127.18.154.62:45053
I20260807 18:17:32.760512 12272 heartbeater.cc:464] Registering TS with master...
I20260807 18:17:32.761282 12272 heartbeater.cc:511] Master 127.18.154.62:45053 requested a full tablet report, sending...
I20260807 18:17:32.763655 12002 ts_manager.cc:194] Registered new tserver with Master: 95b31a70c3184bccad36988f7c4749ff (127.18.154.1:39589)
I20260807 18:17:32.765769 12002 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.154.1:56629
W20260807 18:17:33.105335 12278 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:17:33.105710 12278 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:17:33.105870 12278 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:17:33.113480 12278 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:17:33.113673 12278 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:17:33.113865 12278 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.154.2
I20260807 18:17:33.123417 12278 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.154.20:34085
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126328808105-19048-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126328808105-19048-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.154.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126328808105-19048-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.154.2
--webserver_port=0
--tserver_master_addrs=127.18.154.62:45053
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.154.2
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126328808105-19048-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:17:33.124855 12278 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:17:33.126802 12278 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:17:33.140942 12284 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:17:33.140920 12287 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:17:33.141034 12285 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:17:33.141345 12278 server_base.cc:1034] running on GCE node
I20260807 18:17:33.142139 12278 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:17:33.143371 12278 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:17:33.144604 12278 hybrid_clock.cc:648] HybridClock initialized: now 1786126653144535 us; error 66 us; skew 500 ppm
I20260807 18:17:33.144955 12278 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 18:17:33.147478 12278 webserver.cc:460] Webserver started at http://127.18.154.2:36527/ using document root <none> and password file <none>
I20260807 18:17:33.148214 12278 fs_manager.cc:359] Metadata directory not provided
I20260807 18:17:33.148347 12278 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:17:33.148682 12278 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:17:33.151713 12278 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126328808105-19048-0/minicluster-data/ts-1/data/instance:
uuid: "be84e8c38182406abcd04124f8fc7cf7"
format_stamp: "Formatted at 2026-08-07 18:17:33 on dist-test-slave-gkw7"
server_key: "94be3c94005ddb3e404dbc514e55462b"
server_key_iv: "389fa0f5dda4876d559c2726c68536d0"
server_key_version: "encryptionkey@0"
I20260807 18:17:33.152522 12278 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126328808105-19048-0/minicluster-data/ts-1/wal/instance:
uuid: "be84e8c38182406abcd04124f8fc7cf7"
format_stamp: "Formatted at 2026-08-07 18:17:33 on dist-test-slave-gkw7"
server_key: "94be3c94005ddb3e404dbc514e55462b"
server_key_iv: "389fa0f5dda4876d559c2726c68536d0"
server_key_version: "encryptionkey@0"
I20260807 18:17:33.158516 12278 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260807 18:17:33.162840 12295 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:17:33.164305 12278 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 18:17:33.164453 12278 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126328808105-19048-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126328808105-19048-0/minicluster-data/ts-1/wal
uuid: "be84e8c38182406abcd04124f8fc7cf7"
format_stamp: "Formatted at 2026-08-07 18:17:33 on dist-test-slave-gkw7"
server_key: "94be3c94005ddb3e404dbc514e55462b"
server_key_iv: "389fa0f5dda4876d559c2726c68536d0"
server_key_version: "encryptionkey@0"
I20260807 18:17:33.164646 12278 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126328808105-19048-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126328808105-19048-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126328808105-19048-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:17:33.186774 12278 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:17:33.187825 12278 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:17:33.188212 12278 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:17:33.189352 12278 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:17:33.191473 12278 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:17:33.191619 12278 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:17:33.191756 12278 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:17:33.191844 12278 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:17:33.227325 12278 rpc_server.cc:307] RPC server started. Bound to: 127.18.154.2:42165
I20260807 18:17:33.227439 12455 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.154.2:42165 every 8 connection(s)
I20260807 18:17:33.228736 12278 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126328808105-19048-0/minicluster-data/ts-1/data/info.pb
I20260807 18:17:33.236467 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 12278
I20260807 18:17:33.236661 19048 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126328808105-19048-0/minicluster-data/ts-1/wal/instance
I20260807 18:17:33.237165 19048 external_mini_cluster.cc:1428] Setting key be9416be2a77f1146a67967b647f6c01
I20260807 18:17:33.240434 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126328808105-19048-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126328808105-19048-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126328808105-19048-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126328808105-19048-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.154.3:0
--local_ip_for_outbound_sockets=127.18.154.3
--webserver_interface=127.18.154.3
--webserver_port=0
--tserver_master_addrs=127.18.154.62:45053
--builtin_ntp_servers=127.18.154.20:34085
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 18:17:33.243474 12275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:33.231671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.2:36885 (local address 127.18.154.62:45053)
0807 18:17:33.231879 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:33.231884 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:33.233474 (+ 1590us) server_negotiation.cc:407] Connection header received
0807 18:17:33.235553 (+ 2079us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:33.235560 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:33.235665 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:33.235788 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:17:33.237119 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:33.238206 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:33.240243 (+ 2037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:33.240743 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:33.240850 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:33.242617 (+ 1767us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:17:33.242623 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:17:33.242626 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:17:33.242645 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:17:33.242681 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:17:33.242694 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:17:33.242773 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:33.243131 (+ 358us) server_negotiation.cc:299] Negotiation successful
0807 18:17:33.243222 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260807 18:17:33.244939 12460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:33.232259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:45053 (local address 127.18.154.2:36885)
0807 18:17:33.233301 (+ 1042us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:33.233341 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 18:17:33.235239 (+ 1898us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:17:33.236055 (+ 816us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:17:33.236064 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:17:33.236169 (+ 105us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:17:33.236915 (+ 746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:33.236937 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:33.238369 (+ 1432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:33.238373 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:33.240079 (+ 1706us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:33.240097 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:33.240972 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:33.240981 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:33.241353 (+ 372us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:33.242205 (+ 852us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 18:17:33.242211 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:17:33.242278 (+ 67us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 18:17:33.242398 (+ 120us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:17:33.242796 (+ 398us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:17:33.242809 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:17:33.242820 (+ 11us) client_negotiation.cc:769] Sending connection context
0807 18:17:33.243054 (+ 234us) client_negotiation.cc:240] Negotiation successful
0807 18:17:33.243513 (+ 459us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":493,"thread_start_us":294,"threads_started":1}
I20260807 18:17:33.246568 12456 heartbeater.cc:347] Connected to a master server at 127.18.154.62:45053
I20260807 18:17:33.246930 12456 heartbeater.cc:464] Registering TS with master...
I20260807 18:17:33.247795 12456 heartbeater.cc:511] Master 127.18.154.62:45053 requested a full tablet report, sending...
I20260807 18:17:33.249328 12002 ts_manager.cc:194] Registered new tserver with Master: be84e8c38182406abcd04124f8fc7cf7 (127.18.154.2:42165)
I20260807 18:17:33.250245 12002 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.154.2:36885
W20260807 18:17:33.589910 12461 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:17:33.590276 12461 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:17:33.590447 12461 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:17:33.598114 12461 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:17:33.598240 12461 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:17:33.598392 12461 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.154.3
I20260807 18:17:33.607755 12461 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.154.20:34085
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126328808105-19048-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126328808105-19048-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.154.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126328808105-19048-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.154.3
--webserver_port=0
--tserver_master_addrs=127.18.154.62:45053
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.154.3
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126328808105-19048-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:17:33.609140 12461 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:17:33.611001 12461 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:17:33.624745 12468 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:17:33.624922 12471 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:17:33.625046 12473 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:17:33.626415 12461 server_base.cc:1034] running on GCE node
I20260807 18:17:33.627130 12461 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:17:33.628166 12461 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:17:33.629446 12461 hybrid_clock.cc:648] HybridClock initialized: now 1786126653629375 us; error 60 us; skew 500 ppm
I20260807 18:17:33.629828 12461 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 18:17:33.632251 12461 webserver.cc:460] Webserver started at http://127.18.154.3:46721/ using document root <none> and password file <none>
I20260807 18:17:33.633005 12461 fs_manager.cc:359] Metadata directory not provided
I20260807 18:17:33.633147 12461 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:17:33.633563 12461 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:17:33.636559 12461 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126328808105-19048-0/minicluster-data/ts-2/data/instance:
uuid: "2e7b8e5e9ce1490f8510067ceab56e5a"
format_stamp: "Formatted at 2026-08-07 18:17:33 on dist-test-slave-gkw7"
server_key: "6df28fd69a491a63611459b5c1f6a4e6"
server_key_iv: "fb03330bf0af7e04902e64ba5cac1215"
server_key_version: "encryptionkey@0"
I20260807 18:17:33.637387 12461 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126328808105-19048-0/minicluster-data/ts-2/wal/instance:
uuid: "2e7b8e5e9ce1490f8510067ceab56e5a"
format_stamp: "Formatted at 2026-08-07 18:17:33 on dist-test-slave-gkw7"
server_key: "6df28fd69a491a63611459b5c1f6a4e6"
server_key_iv: "fb03330bf0af7e04902e64ba5cac1215"
server_key_version: "encryptionkey@0"
I20260807 18:17:33.642623 12461 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260807 18:17:33.646282 12479 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:17:33.647578 12461 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 18:17:33.647763 12461 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126328808105-19048-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126328808105-19048-0/minicluster-data/ts-2/wal
uuid: "2e7b8e5e9ce1490f8510067ceab56e5a"
format_stamp: "Formatted at 2026-08-07 18:17:33 on dist-test-slave-gkw7"
server_key: "6df28fd69a491a63611459b5c1f6a4e6"
server_key_iv: "fb03330bf0af7e04902e64ba5cac1215"
server_key_version: "encryptionkey@0"
I20260807 18:17:33.647976 12461 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126328808105-19048-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126328808105-19048-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126328808105-19048-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:17:33.663556 12461 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:17:33.664402 12461 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:17:33.664713 12461 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:17:33.665776 12461 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:17:33.667259 12461 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:17:33.667398 12461 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:17:33.667518 12461 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:17:33.667604 12461 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:17:33.701987 12461 rpc_server.cc:307] RPC server started. Bound to: 127.18.154.3:45365
I20260807 18:17:33.702041 12639 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.154.3:45365 every 8 connection(s)
I20260807 18:17:33.703495 12461 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126328808105-19048-0/minicluster-data/ts-2/data/info.pb
I20260807 18:17:33.703801 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 12461
I20260807 18:17:33.703985 19048 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126328808105-19048-0/minicluster-data/ts-2/wal/instance
I20260807 18:17:33.704556 19048 external_mini_cluster.cc:1428] Setting key 47d8a5fcb06330494b3e739febdc8ecc
I20260807 18:17:33.716780 12275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:33.706534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.3:40621 (local address 127.18.154.62:45053)
0807 18:17:33.706746 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:33.706751 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:33.708639 (+ 1888us) server_negotiation.cc:407] Connection header received
0807 18:17:33.710498 (+ 1859us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:33.710503 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:33.710583 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:33.710681 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:17:33.711773 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:33.712619 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:33.714254 (+ 1635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:33.714675 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:33.714779 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:33.716113 (+ 1334us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:17:33.716117 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:17:33.716121 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:17:33.716139 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:17:33.716170 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:17:33.716183 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:17:33.716264 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:33.716454 (+ 190us) server_negotiation.cc:299] Negotiation successful
0807 18:17:33.716537 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260807 18:17:33.717896 12643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:33.707301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:45053 (local address 127.18.154.3:40621)
0807 18:17:33.708454 (+ 1153us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:33.708510 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0807 18:17:33.710231 (+ 1721us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:17:33.710888 (+ 657us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:17:33.710895 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:17:33.710984 (+ 89us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:17:33.711598 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:33.711615 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:33.712759 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:33.712762 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:33.714082 (+ 1320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:33.714095 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:33.714865 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:33.714869 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:33.715070 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:33.715814 (+ 744us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 18:17:33.715817 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:17:33.715858 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 18:17:33.715938 (+ 80us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:17:33.716274 (+ 336us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:17:33.716282 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:17:33.716288 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 18:17:33.716428 (+ 140us) client_negotiation.cc:240] Negotiation successful
0807 18:17:33.716727 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":581,"thread_start_us":361,"threads_started":1}
I20260807 18:17:33.719583 12640 heartbeater.cc:347] Connected to a master server at 127.18.154.62:45053
I20260807 18:17:33.719919 12640 heartbeater.cc:464] Registering TS with master...
I20260807 18:17:33.720744 12640 heartbeater.cc:511] Master 127.18.154.62:45053 requested a full tablet report, sending...
I20260807 18:17:33.722374 12001 ts_manager.cc:194] Registered new tserver with Master: 2e7b8e5e9ce1490f8510067ceab56e5a (127.18.154.3:45365)
I20260807 18:17:33.723351 12001 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.154.3:40621
I20260807 18:17:33.732877 19048 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 18:17:33.743273 12275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:33.737268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57544 (local address 127.18.154.62:45053)
0807 18:17:33.737564 (+ 296us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:33.737572 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:33.737911 (+ 339us) server_negotiation.cc:407] Connection header received
0807 18:17:33.738054 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:33.738058 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:33.738137 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:33.738232 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:17:33.738754 (+ 522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:33.739462 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:33.740843 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:33.741259 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:33.741351 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:33.742558 (+ 1207us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:17:33.742564 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:17:33.742569 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:17:33.742589 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:17:33.742629 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:17:33.742644 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:17:33.742751 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:33.742861 (+ 110us) server_negotiation.cc:299] Negotiation successful
0807 18:17:33.742962 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260807 18:17:33.747689 12001 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:57544:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 18:17:33.754810 12001 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:17:33.768929 12272 heartbeater.cc:503] Master 127.18.154.62:45053 was elected leader, sending a full tablet report...
I20260807 18:17:33.798180 12663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:33.770057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38162 (local address 127.18.154.2:42165)
0807 18:17:33.790006 (+ 19949us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:33.790017 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:33.790809 (+ 792us) server_negotiation.cc:407] Connection header received
0807 18:17:33.790912 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:33.790920 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:33.791202 (+ 282us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:33.791412 (+ 210us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:17:33.792165 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:33.793636 (+ 1471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:33.795743 (+ 2107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:33.796845 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:33.796993 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:33.797298 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:33.797617 (+ 319us) server_negotiation.cc:299] Negotiation successful
0807 18:17:33.797803 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19765,"thread_start_us":19637,"threads_started":1}
I20260807 18:17:33.798530 12662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:33.769342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.2:42165 (local address 127.0.0.1:38162)
0807 18:17:33.790505 (+ 21163us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:33.790550 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 18:17:33.790677 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:17:33.791463 (+ 786us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:17:33.791471 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:17:33.791511 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:17:33.791985 (+ 474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:33.792003 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:33.793817 (+ 1814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:33.793825 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:33.795445 (+ 1620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:33.795458 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:33.797084 (+ 1626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:33.797091 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:33.797367 (+ 276us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:33.797408 (+ 41us) client_negotiation.cc:769] Sending connection context
0807 18:17:33.797567 (+ 159us) client_negotiation.cc:240] Negotiation successful
0807 18:17:33.797850 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21026,"mutex_wait_us":49,"thread_start_us":20865,"threads_started":1}
I20260807 18:17:33.798612 12660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:33.768137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.3:45365 (local address 127.0.0.1:54030)
0807 18:17:33.788456 (+ 20319us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:33.788507 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0807 18:17:33.788696 (+ 189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:17:33.790097 (+ 1401us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:17:33.790109 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:17:33.790162 (+ 53us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:17:33.790689 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:33.790701 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:33.793022 (+ 2321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:33.793027 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:33.795755 (+ 2728us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:33.795771 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:33.797794 (+ 2023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:33.797801 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:33.798023 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:33.798055 (+ 32us) client_negotiation.cc:769] Sending connection context
0807 18:17:33.798139 (+ 84us) client_negotiation.cc:240] Negotiation successful
0807 18:17:33.798279 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20028,"thread_start_us":20909,"threads_started":1}
I20260807 18:17:33.799788 12657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:33.768391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54030 (local address 127.18.154.3:45365)
0807 18:17:33.787907 (+ 19516us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:33.787922 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:33.789337 (+ 1415us) server_negotiation.cc:407] Connection header received
0807 18:17:33.789468 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:33.789477 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:33.789824 (+ 347us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:33.790926 (+ 1102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:17:33.791203 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:33.792849 (+ 1646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:33.795976 (+ 3127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:33.797524 (+ 1548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:33.798950 (+ 1426us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:33.799171 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:33.799237 (+ 66us) server_negotiation.cc:299] Negotiation successful
0807 18:17:33.799434 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19236,"thread_start_us":19027,"threads_started":1}
I20260807 18:17:33.800778 12659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:33.768356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.1:39589 (local address 127.0.0.1:38666)
0807 18:17:33.789340 (+ 20984us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:33.789360 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 18:17:33.789466 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:17:33.792620 (+ 3154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:17:33.792624 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:17:33.792651 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:17:33.793009 (+ 358us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:33.793028 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:33.795588 (+ 2560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:33.795595 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:33.797802 (+ 2207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:33.797814 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:33.799676 (+ 1862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:33.799682 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:33.799908 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:33.799949 (+ 41us) client_negotiation.cc:769] Sending connection context
0807 18:17:33.800053 (+ 104us) client_negotiation.cc:240] Negotiation successful
0807 18:17:33.800181 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20848,"mutex_wait_us":3,"thread_start_us":19646,"threads_started":1}
I20260807 18:17:33.802232 12661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:33.768799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38666 (local address 127.18.154.1:39589)
0807 18:17:33.792048 (+ 23249us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:33.792056 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:33.792083 (+ 27us) server_negotiation.cc:407] Connection header received
0807 18:17:33.792181 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:33.792188 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:33.792417 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:33.792597 (+ 180us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:17:33.793863 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:33.795444 (+ 1581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:33.798037 (+ 2593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:33.799484 (+ 1447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:33.799641 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:33.801652 (+ 2011us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:33.801742 (+ 90us) server_negotiation.cc:299] Negotiation successful
0807 18:17:33.801923 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23070,"thread_start_us":22937,"threads_started":1}
I20260807 18:17:33.804037 12550 tablet_service.cc:1467] Processing CreateTablet for tablet a6e5393ee68f44ee9cc6ee09883433f3 (DEFAULT_TABLE table=test-table [id=ca00a2aa7f7d4981a880e2aa7dc77ab9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:17:33.804037 12368 tablet_service.cc:1467] Processing CreateTablet for tablet a6e5393ee68f44ee9cc6ee09883433f3 (DEFAULT_TABLE table=test-table [id=ca00a2aa7f7d4981a880e2aa7dc77ab9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:17:33.805912 12368 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a6e5393ee68f44ee9cc6ee09883433f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:17:33.805912 12550 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a6e5393ee68f44ee9cc6ee09883433f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:17:33.805456 12185 tablet_service.cc:1467] Processing CreateTablet for tablet a6e5393ee68f44ee9cc6ee09883433f3 (DEFAULT_TABLE table=test-table [id=ca00a2aa7f7d4981a880e2aa7dc77ab9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:17:33.806771 12185 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a6e5393ee68f44ee9cc6ee09883433f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:17:33.818372 12664 tablet_bootstrap.cc:492] T a6e5393ee68f44ee9cc6ee09883433f3 P 95b31a70c3184bccad36988f7c4749ff: Bootstrap starting.
I20260807 18:17:33.822575 12666 tablet_bootstrap.cc:492] T a6e5393ee68f44ee9cc6ee09883433f3 P be84e8c38182406abcd04124f8fc7cf7: Bootstrap starting.
I20260807 18:17:33.822932 12664 tablet_bootstrap.cc:654] T a6e5393ee68f44ee9cc6ee09883433f3 P 95b31a70c3184bccad36988f7c4749ff: Neither blocks nor log segments found. Creating new log.
I20260807 18:17:33.824182 12667 tablet_bootstrap.cc:492] T a6e5393ee68f44ee9cc6ee09883433f3 P 2e7b8e5e9ce1490f8510067ceab56e5a: Bootstrap starting.
I20260807 18:17:33.824519 12664 log.cc:824] T a6e5393ee68f44ee9cc6ee09883433f3 P 95b31a70c3184bccad36988f7c4749ff: Log is configured to *not* fsync() on all Append() calls
I20260807 18:17:33.826742 12666 tablet_bootstrap.cc:654] T a6e5393ee68f44ee9cc6ee09883433f3 P be84e8c38182406abcd04124f8fc7cf7: Neither blocks nor log segments found. Creating new log.
I20260807 18:17:33.827431 12664 tablet_bootstrap.cc:492] T a6e5393ee68f44ee9cc6ee09883433f3 P 95b31a70c3184bccad36988f7c4749ff: No bootstrap required, opened a new log
I20260807 18:17:33.827725 12664 ts_tablet_manager.cc:1397] T a6e5393ee68f44ee9cc6ee09883433f3 P 95b31a70c3184bccad36988f7c4749ff: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260807 18:17:33.828166 12667 tablet_bootstrap.cc:654] T a6e5393ee68f44ee9cc6ee09883433f3 P 2e7b8e5e9ce1490f8510067ceab56e5a: Neither blocks nor log segments found. Creating new log.
I20260807 18:17:33.828188 12666 log.cc:824] T a6e5393ee68f44ee9cc6ee09883433f3 P be84e8c38182406abcd04124f8fc7cf7: Log is configured to *not* fsync() on all Append() calls
I20260807 18:17:33.829454 12667 log.cc:824] T a6e5393ee68f44ee9cc6ee09883433f3 P 2e7b8e5e9ce1490f8510067ceab56e5a: Log is configured to *not* fsync() on all Append() calls
I20260807 18:17:33.831660 12666 tablet_bootstrap.cc:492] T a6e5393ee68f44ee9cc6ee09883433f3 P be84e8c38182406abcd04124f8fc7cf7: No bootstrap required, opened a new log
I20260807 18:17:33.831988 12666 ts_tablet_manager.cc:1397] T a6e5393ee68f44ee9cc6ee09883433f3 P be84e8c38182406abcd04124f8fc7cf7: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260807 18:17:33.832820 12667 tablet_bootstrap.cc:492] T a6e5393ee68f44ee9cc6ee09883433f3 P 2e7b8e5e9ce1490f8510067ceab56e5a: No bootstrap required, opened a new log
I20260807 18:17:33.833109 12667 ts_tablet_manager.cc:1397] T a6e5393ee68f44ee9cc6ee09883433f3 P 2e7b8e5e9ce1490f8510067ceab56e5a: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260807 18:17:33.836377 12664 raft_consensus.cc:348] T a6e5393ee68f44ee9cc6ee09883433f3 P 95b31a70c3184bccad36988f7c4749ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e7b8e5e9ce1490f8510067ceab56e5a" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 45365 } } peers { permanent_uuid: "95b31a70c3184bccad36988f7c4749ff" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 39589 } } peers { permanent_uuid: "be84e8c38182406abcd04124f8fc7cf7" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 42165 } }
I20260807 18:17:33.836763 12664 raft_consensus.cc:374] T a6e5393ee68f44ee9cc6ee09883433f3 P 95b31a70c3184bccad36988f7c4749ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:17:33.836892 12664 raft_consensus.cc:729] T a6e5393ee68f44ee9cc6ee09883433f3 P 95b31a70c3184bccad36988f7c4749ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95b31a70c3184bccad36988f7c4749ff, State: Initialized, Role: FOLLOWER
I20260807 18:17:33.837489 12664 consensus_queue.cc:260] T a6e5393ee68f44ee9cc6ee09883433f3 P 95b31a70c3184bccad36988f7c4749ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e7b8e5e9ce1490f8510067ceab56e5a" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 45365 } } peers { permanent_uuid: "95b31a70c3184bccad36988f7c4749ff" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 39589 } } peers { permanent_uuid: "be84e8c38182406abcd04124f8fc7cf7" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 42165 } }
I20260807 18:17:33.839306 12664 ts_tablet_manager.cc:1428] T a6e5393ee68f44ee9cc6ee09883433f3 P 95b31a70c3184bccad36988f7c4749ff: Time spent starting tablet: real 0.011s user 0.012s sys 0.000s
I20260807 18:17:33.845115 12667 raft_consensus.cc:348] T a6e5393ee68f44ee9cc6ee09883433f3 P 2e7b8e5e9ce1490f8510067ceab56e5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e7b8e5e9ce1490f8510067ceab56e5a" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 45365 } } peers { permanent_uuid: "95b31a70c3184bccad36988f7c4749ff" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 39589 } } peers { permanent_uuid: "be84e8c38182406abcd04124f8fc7cf7" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 42165 } }
I20260807 18:17:33.845014 12666 raft_consensus.cc:348] T a6e5393ee68f44ee9cc6ee09883433f3 P be84e8c38182406abcd04124f8fc7cf7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e7b8e5e9ce1490f8510067ceab56e5a" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 45365 } } peers { permanent_uuid: "95b31a70c3184bccad36988f7c4749ff" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 39589 } } peers { permanent_uuid: "be84e8c38182406abcd04124f8fc7cf7" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 42165 } }
I20260807 18:17:33.845611 12666 raft_consensus.cc:374] T a6e5393ee68f44ee9cc6ee09883433f3 P be84e8c38182406abcd04124f8fc7cf7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:17:33.845619 12667 raft_consensus.cc:374] T a6e5393ee68f44ee9cc6ee09883433f3 P 2e7b8e5e9ce1490f8510067ceab56e5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:17:33.845738 12666 raft_consensus.cc:729] T a6e5393ee68f44ee9cc6ee09883433f3 P be84e8c38182406abcd04124f8fc7cf7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be84e8c38182406abcd04124f8fc7cf7, State: Initialized, Role: FOLLOWER
I20260807 18:17:33.845738 12667 raft_consensus.cc:729] T a6e5393ee68f44ee9cc6ee09883433f3 P 2e7b8e5e9ce1490f8510067ceab56e5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e7b8e5e9ce1490f8510067ceab56e5a, State: Initialized, Role: FOLLOWER
I20260807 18:17:33.846427 12666 consensus_queue.cc:260] T a6e5393ee68f44ee9cc6ee09883433f3 P be84e8c38182406abcd04124f8fc7cf7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e7b8e5e9ce1490f8510067ceab56e5a" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 45365 } } peers { permanent_uuid: "95b31a70c3184bccad36988f7c4749ff" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 39589 } } peers { permanent_uuid: "be84e8c38182406abcd04124f8fc7cf7" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 42165 } }
I20260807 18:17:33.846421 12667 consensus_queue.cc:260] T a6e5393ee68f44ee9cc6ee09883433f3 P 2e7b8e5e9ce1490f8510067ceab56e5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e7b8e5e9ce1490f8510067ceab56e5a" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 45365 } } peers { permanent_uuid: "95b31a70c3184bccad36988f7c4749ff" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 39589 } } peers { permanent_uuid: "be84e8c38182406abcd04124f8fc7cf7" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 42165 } }
I20260807 18:17:33.847600 12456 heartbeater.cc:503] Master 127.18.154.62:45053 was elected leader, sending a full tablet report...
I20260807 18:17:33.848133 12640 heartbeater.cc:503] Master 127.18.154.62:45053 was elected leader, sending a full tablet report...
I20260807 18:17:33.848546 12666 ts_tablet_manager.cc:1428] T a6e5393ee68f44ee9cc6ee09883433f3 P be84e8c38182406abcd04124f8fc7cf7: Time spent starting tablet: real 0.016s user 0.012s sys 0.005s
I20260807 18:17:33.849730 12667 ts_tablet_manager.cc:1428] T a6e5393ee68f44ee9cc6ee09883433f3 P 2e7b8e5e9ce1490f8510067ceab56e5a: Time spent starting tablet: real 0.016s user 0.014s sys 0.004s
W20260807 18:17:33.955509 12642 tablet.cc:2369] T a6e5393ee68f44ee9cc6ee09883433f3 P 2e7b8e5e9ce1490f8510067ceab56e5a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 18:17:33.981073 12457 tablet.cc:2369] T a6e5393ee68f44ee9cc6ee09883433f3 P be84e8c38182406abcd04124f8fc7cf7: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 18:17:33.996712 12273 tablet.cc:2369] T a6e5393ee68f44ee9cc6ee09883433f3 P 95b31a70c3184bccad36988f7c4749ff: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:17:34.070010 12673 raft_consensus.cc:482] T a6e5393ee68f44ee9cc6ee09883433f3 P 2e7b8e5e9ce1490f8510067ceab56e5a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:17:34.070381 12673 raft_consensus.cc:504] T a6e5393ee68f44ee9cc6ee09883433f3 P 2e7b8e5e9ce1490f8510067ceab56e5a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e7b8e5e9ce1490f8510067ceab56e5a" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 45365 } } peers { permanent_uuid: "95b31a70c3184bccad36988f7c4749ff" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 39589 } } peers { permanent_uuid: "be84e8c38182406abcd04124f8fc7cf7" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 42165 } }
I20260807 18:17:34.072199 12673 leader_election.cc:288] T a6e5393ee68f44ee9cc6ee09883433f3 P 2e7b8e5e9ce1490f8510067ceab56e5a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 95b31a70c3184bccad36988f7c4749ff (127.18.154.1:39589), be84e8c38182406abcd04124f8fc7cf7 (127.18.154.2:42165)
I20260807 18:17:34.079042 12643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:34.072404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.1:39589 (local address 127.18.154.3:53415)
0807 18:17:34.072643 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:34.072669 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0807 18:17:34.072824 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:17:34.073240 (+ 416us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:17:34.073248 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:17:34.073290 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:17:34.073611 (+ 321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:34.073624 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:34.074971 (+ 1347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:34.074975 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:34.076855 (+ 1880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:34.076871 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:34.078385 (+ 1514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:34.078389 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:34.078521 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:34.078547 (+ 26us) client_negotiation.cc:769] Sending connection context
0807 18:17:34.078621 (+ 74us) client_negotiation.cc:240] Negotiation successful
0807 18:17:34.078728 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"mutex_wait_us":25}
I20260807 18:17:34.079066 12661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:34.072615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.3:53415 (local address 127.18.154.1:39589)
0807 18:17:34.072834 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:34.072841 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:34.072865 (+ 24us) server_negotiation.cc:407] Connection header received
0807 18:17:34.072952 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:34.072957 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:34.073082 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:34.073201 (+ 119us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:17:34.073834 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:34.074838 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:34.077105 (+ 2267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:34.078185 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:34.078376 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:34.078451 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:34.078671 (+ 220us) server_negotiation.cc:299] Negotiation successful
0807 18:17:34.078806 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260807 18:17:34.079763 12663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:34.072895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.3:43451 (local address 127.18.154.2:42165)
0807 18:17:34.073071 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:34.073075 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:34.073838 (+ 763us) server_negotiation.cc:407] Connection header received
0807 18:17:34.073952 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:34.073959 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:34.074053 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:34.074158 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:17:34.074657 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:34.075453 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:34.077615 (+ 2162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:34.078792 (+ 1177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:34.078942 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:34.079003 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:34.079371 (+ 368us) server_negotiation.cc:299] Negotiation successful
0807 18:17:34.079480 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260807 18:17:34.080169 12211 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6e5393ee68f44ee9cc6ee09883433f3" candidate_uuid: "2e7b8e5e9ce1490f8510067ceab56e5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95b31a70c3184bccad36988f7c4749ff" is_pre_election: true
I20260807 18:17:34.080498 12679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:34.072806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.2:42165 (local address 127.18.154.3:43451)
0807 18:17:34.073714 (+ 908us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:34.073732 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 18:17:34.073848 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:17:34.074171 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:17:34.074174 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:17:34.074193 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:17:34.074490 (+ 297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:34.074498 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:34.075615 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:34.075621 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:34.077415 (+ 1794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:34.077426 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:34.078965 (+ 1539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:34.078970 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:34.079169 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:34.079205 (+ 36us) client_negotiation.cc:769] Sending connection context
0807 18:17:34.079957 (+ 752us) client_negotiation.cc:240] Negotiation successful
0807 18:17:34.080092 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":774,"mutex_wait_us":4,"thread_start_us":536,"threads_started":1}
I20260807 18:17:34.080783 12211 raft_consensus.cc:2454] T a6e5393ee68f44ee9cc6ee09883433f3 P 95b31a70c3184bccad36988f7c4749ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2e7b8e5e9ce1490f8510067ceab56e5a in term 0.
I20260807 18:17:34.081503 12486 leader_election.cc:302] T a6e5393ee68f44ee9cc6ee09883433f3 P 2e7b8e5e9ce1490f8510067ceab56e5a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2e7b8e5e9ce1490f8510067ceab56e5a, 95b31a70c3184bccad36988f7c4749ff; no voters:
I20260807 18:17:34.081662 12398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6e5393ee68f44ee9cc6ee09883433f3" candidate_uuid: "2e7b8e5e9ce1490f8510067ceab56e5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be84e8c38182406abcd04124f8fc7cf7" is_pre_election: true
I20260807 18:17:34.082016 12673 raft_consensus.cc:2781] T a6e5393ee68f44ee9cc6ee09883433f3 P 2e7b8e5e9ce1490f8510067ceab56e5a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:17:34.082175 12673 raft_consensus.cc:482] T a6e5393ee68f44ee9cc6ee09883433f3 P 2e7b8e5e9ce1490f8510067ceab56e5a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:17:34.082309 12673 raft_consensus.cc:3037] T a6e5393ee68f44ee9cc6ee09883433f3 P 2e7b8e5e9ce1490f8510067ceab56e5a [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:17:34.082314 12398 raft_consensus.cc:2454] T a6e5393ee68f44ee9cc6ee09883433f3 P be84e8c38182406abcd04124f8fc7cf7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2e7b8e5e9ce1490f8510067ceab56e5a in term 0.
I20260807 18:17:34.084616 12673 raft_consensus.cc:504] T a6e5393ee68f44ee9cc6ee09883433f3 P 2e7b8e5e9ce1490f8510067ceab56e5a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e7b8e5e9ce1490f8510067ceab56e5a" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 45365 } } peers { permanent_uuid: "95b31a70c3184bccad36988f7c4749ff" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 39589 } } peers { permanent_uuid: "be84e8c38182406abcd04124f8fc7cf7" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 42165 } }
I20260807 18:17:34.085273 12673 leader_election.cc:288] T a6e5393ee68f44ee9cc6ee09883433f3 P 2e7b8e5e9ce1490f8510067ceab56e5a [CANDIDATE]: Term 1 election: Requested vote from peers 95b31a70c3184bccad36988f7c4749ff (127.18.154.1:39589), be84e8c38182406abcd04124f8fc7cf7 (127.18.154.2:42165)
I20260807 18:17:34.085680 12211 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6e5393ee68f44ee9cc6ee09883433f3" candidate_uuid: "2e7b8e5e9ce1490f8510067ceab56e5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95b31a70c3184bccad36988f7c4749ff"
I20260807 18:17:34.085754 12398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6e5393ee68f44ee9cc6ee09883433f3" candidate_uuid: "2e7b8e5e9ce1490f8510067ceab56e5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be84e8c38182406abcd04124f8fc7cf7"
I20260807 18:17:34.085888 12211 raft_consensus.cc:3037] T a6e5393ee68f44ee9cc6ee09883433f3 P 95b31a70c3184bccad36988f7c4749ff [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:17:34.085947 12398 raft_consensus.cc:3037] T a6e5393ee68f44ee9cc6ee09883433f3 P be84e8c38182406abcd04124f8fc7cf7 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:17:34.088131 12211 raft_consensus.cc:2454] T a6e5393ee68f44ee9cc6ee09883433f3 P 95b31a70c3184bccad36988f7c4749ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2e7b8e5e9ce1490f8510067ceab56e5a in term 1.
I20260807 18:17:34.088138 12398 raft_consensus.cc:2454] T a6e5393ee68f44ee9cc6ee09883433f3 P be84e8c38182406abcd04124f8fc7cf7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2e7b8e5e9ce1490f8510067ceab56e5a in term 1.
I20260807 18:17:34.088523 12486 leader_election.cc:302] T a6e5393ee68f44ee9cc6ee09883433f3 P 2e7b8e5e9ce1490f8510067ceab56e5a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2e7b8e5e9ce1490f8510067ceab56e5a, 95b31a70c3184bccad36988f7c4749ff; no voters:
I20260807 18:17:34.088856 12673 raft_consensus.cc:2781] T a6e5393ee68f44ee9cc6ee09883433f3 P 2e7b8e5e9ce1490f8510067ceab56e5a [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:17:34.089287 12673 raft_consensus.cc:686] T a6e5393ee68f44ee9cc6ee09883433f3 P 2e7b8e5e9ce1490f8510067ceab56e5a [term 1 LEADER]: Becoming Leader. State: Replica: 2e7b8e5e9ce1490f8510067ceab56e5a, State: Running, Role: LEADER
I20260807 18:17:34.089890 12673 consensus_queue.cc:237] T a6e5393ee68f44ee9cc6ee09883433f3 P 2e7b8e5e9ce1490f8510067ceab56e5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e7b8e5e9ce1490f8510067ceab56e5a" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 45365 } } peers { permanent_uuid: "95b31a70c3184bccad36988f7c4749ff" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 39589 } } peers { permanent_uuid: "be84e8c38182406abcd04124f8fc7cf7" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 42165 } }
I20260807 18:17:34.094498 12002 catalog_manager.cc:5495] T a6e5393ee68f44ee9cc6ee09883433f3 P 2e7b8e5e9ce1490f8510067ceab56e5a reported cstate change: term changed from 0 to 1, leader changed from <none> to 2e7b8e5e9ce1490f8510067ceab56e5a (127.18.154.3). New cstate: current_term: 1 leader_uuid: "2e7b8e5e9ce1490f8510067ceab56e5a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e7b8e5e9ce1490f8510067ceab56e5a" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 45365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "95b31a70c3184bccad36988f7c4749ff" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 39589 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "be84e8c38182406abcd04124f8fc7cf7" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 42165 } health_report { overall_health: UNKNOWN } } }
I20260807 18:17:34.149791 12657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:34.143639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54036 (local address 127.18.154.3:45365)
0807 18:17:34.143855 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:34.143861 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:34.143882 (+ 21us) server_negotiation.cc:407] Connection header received
0807 18:17:34.144091 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:34.144096 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:34.144194 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:34.144360 (+ 166us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:17:34.144687 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:34.145843 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:34.147251 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:34.147896 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:34.148017 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:34.148515 (+ 498us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:17:34.149225 (+ 710us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:17:34.149374 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:34.149424 (+ 50us) server_negotiation.cc:299] Negotiation successful
0807 18:17:34.149512 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260807 18:17:34.158814 12398 raft_consensus.cc:1261] T a6e5393ee68f44ee9cc6ee09883433f3 P be84e8c38182406abcd04124f8fc7cf7 [term 1 FOLLOWER]: Refusing update from remote peer 2e7b8e5e9ce1490f8510067ceab56e5a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:17:34.158957 12211 raft_consensus.cc:1261] T a6e5393ee68f44ee9cc6ee09883433f3 P 95b31a70c3184bccad36988f7c4749ff [term 1 FOLLOWER]: Refusing update from remote peer 2e7b8e5e9ce1490f8510067ceab56e5a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:17:34.159868 12680 consensus_queue.cc:1035] T a6e5393ee68f44ee9cc6ee09883433f3 P 2e7b8e5e9ce1490f8510067ceab56e5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "be84e8c38182406abcd04124f8fc7cf7" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 42165 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:17:34.160359 12673 consensus_queue.cc:1035] T a6e5393ee68f44ee9cc6ee09883433f3 P 2e7b8e5e9ce1490f8510067ceab56e5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "95b31a70c3184bccad36988f7c4749ff" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 39589 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:17:34.177830 12685 mvcc.cc:205] Tried to move back new op lower bound from 7315974775424569344 to 7315974775155879936. Current Snapshot: MvccSnapshot[applied={T|T < 7315974775424569344}]
I20260807 18:17:34.189858 12002 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:57544:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:17:34.190207 12002 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:57544:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:17:34.193799 12002 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 79210462dc754603b9148cfb6d4426ba: Sending DeleteTablet for 3 replicas of tablet a6e5393ee68f44ee9cc6ee09883433f3
I20260807 18:17:34.195011 12185 tablet_service.cc:1514] Processing DeleteTablet for tablet a6e5393ee68f44ee9cc6ee09883433f3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:17:34 UTC) from {username='slave'} at 127.0.0.1:38666
I20260807 18:17:34.195245 12550 tablet_service.cc:1514] Processing DeleteTablet for tablet a6e5393ee68f44ee9cc6ee09883433f3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:17:34 UTC) from {username='slave'} at 127.0.0.1:54030
I20260807 18:17:34.195546 12368 tablet_service.cc:1514] Processing DeleteTablet for tablet a6e5393ee68f44ee9cc6ee09883433f3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:17:34 UTC) from {username='slave'} at 127.0.0.1:38162
I20260807 18:17:34.197422 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 12085
I20260807 18:17:34.203506 12697 tablet_replica.cc:330] T a6e5393ee68f44ee9cc6ee09883433f3 P be84e8c38182406abcd04124f8fc7cf7: stopping tablet replica
I20260807 18:17:34.204315 12697 raft_consensus.cc:2229] T a6e5393ee68f44ee9cc6ee09883433f3 P be84e8c38182406abcd04124f8fc7cf7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:17:34.204964 12697 raft_consensus.cc:2258] T a6e5393ee68f44ee9cc6ee09883433f3 P be84e8c38182406abcd04124f8fc7cf7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:17:34.207844 12697 ts_tablet_manager.cc:1905] T a6e5393ee68f44ee9cc6ee09883433f3 P be84e8c38182406abcd04124f8fc7cf7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:17:34.218137 12695 tablet_replica.cc:330] T a6e5393ee68f44ee9cc6ee09883433f3 P 2e7b8e5e9ce1490f8510067ceab56e5a: stopping tablet replica
I20260807 18:17:34.218612 12695 raft_consensus.cc:2229] T a6e5393ee68f44ee9cc6ee09883433f3 P 2e7b8e5e9ce1490f8510067ceab56e5a [term 1 LEADER]: Raft consensus shutting down.
I20260807 18:17:34.219365 12695 raft_consensus.cc:2258] T a6e5393ee68f44ee9cc6ee09883433f3 P 2e7b8e5e9ce1490f8510067ceab56e5a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:17:34.221571 12697 ts_tablet_manager.cc:1918] T a6e5393ee68f44ee9cc6ee09883433f3 P be84e8c38182406abcd04124f8fc7cf7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:17:34.221834 12697 log.cc:1196] T a6e5393ee68f44ee9cc6ee09883433f3 P be84e8c38182406abcd04124f8fc7cf7: Deleting WAL directory at /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126328808105-19048-0/minicluster-data/ts-1/wal/wals/a6e5393ee68f44ee9cc6ee09883433f3
I20260807 18:17:34.222007 12695 ts_tablet_manager.cc:1905] T a6e5393ee68f44ee9cc6ee09883433f3 P 2e7b8e5e9ce1490f8510067ceab56e5a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:17:34.222584 12697 ts_tablet_manager.cc:1939] T a6e5393ee68f44ee9cc6ee09883433f3 P be84e8c38182406abcd04124f8fc7cf7: Deleting consensus metadata
I20260807 18:17:34.224579 11982 catalog_manager.cc:4841] TS be84e8c38182406abcd04124f8fc7cf7 (127.18.154.2:42165): tablet a6e5393ee68f44ee9cc6ee09883433f3 (table test-table [id=ca00a2aa7f7d4981a880e2aa7dc77ab9]) successfully deleted
I20260807 18:17:34.225461 12693 tablet_replica.cc:330] T a6e5393ee68f44ee9cc6ee09883433f3 P 95b31a70c3184bccad36988f7c4749ff: stopping tablet replica
I20260807 18:17:34.226063 12693 raft_consensus.cc:2229] T a6e5393ee68f44ee9cc6ee09883433f3 P 95b31a70c3184bccad36988f7c4749ff [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:17:34.227289 12661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:34.198680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38674 (local address 127.18.154.1:39589)
0807 18:17:34.198874 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:34.198880 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:34.216384 (+ 17504us) server_negotiation.cc:407] Connection header received
0807 18:17:34.216913 (+ 529us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:34.216919 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:34.217037 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:34.217118 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:17:34.218044 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:34.219805 (+ 1761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:34.222581 (+ 2776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:34.223536 (+ 955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:34.223665 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:34.225451 (+ 1786us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:17:34.225464 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:17:34.225482 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:17:34.226352 (+ 870us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:17:34.226417 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:17:34.226443 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:17:34.226549 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:34.226808 (+ 259us) server_negotiation.cc:299] Negotiation successful
0807 18:17:34.226929 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260807 18:17:34.228474 12265 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:17:34.229318 12695 ts_tablet_manager.cc:1918] T a6e5393ee68f44ee9cc6ee09883433f3 P 2e7b8e5e9ce1490f8510067ceab56e5a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:17:34.229485 12695 log.cc:1196] T a6e5393ee68f44ee9cc6ee09883433f3 P 2e7b8e5e9ce1490f8510067ceab56e5a: Deleting WAL directory at /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786126328808105-19048-0/minicluster-data/ts-2/wal/wals/a6e5393ee68f44ee9cc6ee09883433f3
I20260807 18:17:34.230063 12695 ts_tablet_manager.cc:1939] T a6e5393ee68f44ee9cc6ee09883433f3 P 2e7b8e5e9ce1490f8510067ceab56e5a: Deleting consensus metadata
I20260807 18:17:34.231822 11980 catalog_manager.cc:4841] TS 2e7b8e5e9ce1490f8510067ceab56e5a (127.18.154.3:45365): tablet a6e5393ee68f44ee9cc6ee09883433f3 (table test-table [id=ca00a2aa7f7d4981a880e2aa7dc77ab9]) successfully deleted
I20260807 18:17:34.329625 12693 raft_consensus.cc:2258] T a6e5393ee68f44ee9cc6ee09883433f3 P 95b31a70c3184bccad36988f7c4749ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:17:34.330592 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 12085
W20260807 18:17:34.354738 11983 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.154.1:39589 (error 108)
I20260807 18:17:34.355082 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 12278
I20260807 18:17:34.357144 12705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:34.356355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.1:39589 (local address 127.0.0.1:38676)
0807 18:17:34.356813 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:34.356924 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.1:39589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":334,"thread_start_us":242,"threads_started":1}
W20260807 18:17:34.357425 11983 catalog_manager.cc:4584] TS 95b31a70c3184bccad36988f7c4749ff (127.18.154.1:39589): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a6e5393ee68f44ee9cc6ee09883433f3: Network error: Client connection negotiation failed: client connection to 127.18.154.1:39589: connect: Connection refused (error 111)
I20260807 18:17:34.364270 12663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:34.356456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38178 (local address 127.18.154.2:42165)
0807 18:17:34.356667 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:34.356674 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:34.356697 (+ 23us) server_negotiation.cc:407] Connection header received
0807 18:17:34.356769 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:34.356774 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:34.356888 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:34.357013 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:17:34.357699 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:34.358963 (+ 1264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:34.360398 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:34.361135 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:34.361265 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:34.362517 (+ 1252us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:17:34.362530 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:17:34.362560 (+ 30us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:17:34.363409 (+ 849us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:17:34.363477 (+ 68us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:17:34.363504 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:17:34.363642 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:34.363747 (+ 105us) server_negotiation.cc:299] Negotiation successful
0807 18:17:34.363865 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260807 18:17:34.364971 12451 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:17:34.411446 12705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:34.410813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.1:39589 (local address 127.0.0.1:38690)
0807 18:17:34.411109 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:34.411257 (+ 148us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.1:39589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125}
I20260807 18:17:34.412770 12705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:34.412285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.1:39589 (local address 127.0.0.1:38694)
0807 18:17:34.412471 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:34.412569 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.1:39589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260807 18:17:34.456058 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 12278
I20260807 18:17:34.473485 12705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:34.472869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.1:39589 (local address 127.0.0.1:38708)
0807 18:17:34.473143 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:34.473274 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.1:39589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260807 18:17:34.475068 12705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:34.474522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.1:39589 (local address 127.0.0.1:38718)
0807 18:17:34.474740 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:34.474842 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.1:39589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260807 18:17:34.482795 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 12461
I20260807 18:17:34.491340 12657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:34.483798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54046 (local address 127.18.154.3:45365)
0807 18:17:34.483960 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:34.483965 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:34.483982 (+ 17us) server_negotiation.cc:407] Connection header received
0807 18:17:34.484087 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:34.484090 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:34.484166 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:34.484273 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:17:34.484841 (+ 568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:34.485831 (+ 990us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:34.488005 (+ 2174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:34.488435 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:34.488534 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:34.489860 (+ 1326us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:17:34.489870 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:17:34.489886 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:17:34.490510 (+ 624us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:17:34.490561 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:17:34.490588 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:17:34.490677 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:34.490864 (+ 187us) server_negotiation.cc:299] Negotiation successful
0807 18:17:34.490964 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 18:17:34.492066 12629 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:17:34.555936 12705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:34.555418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.1:39589 (local address 127.0.0.1:38728)
0807 18:17:34.555680 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:34.555788 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.1:39589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98}
I20260807 18:17:34.557092 12705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:34.556634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.1:39589 (local address 127.0.0.1:38736)
0807 18:17:34.556862 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:34.556934 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.1:39589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119}
I20260807 18:17:34.617153 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 12461
I20260807 18:17:34.645303 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 11958
I20260807 18:17:34.646673 12043 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:17:34.756913 12705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:34.756389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.1:39589 (local address 127.0.0.1:38746)
0807 18:17:34.756583 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:34.756730 (+ 147us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.1:39589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260807 18:17:34.756915 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 11958
2026-08-07T18:17:34Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (3171 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126328808105-19048-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:17:34 dist-test-slave-gkw7 krb5kdc[12715](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:17:34 dist-test-slave-gkw7 krb5kdc[12715](info): set up 2 sockets
Aug 07 18:17:34 dist-test-slave-gkw7 krb5kdc[12715](info): commencing operation
krb5kdc: starting...
W20260807 18:17:36.058461 19048 mini_kdc.cc:121] Time spent starting KDC: real 1.253s user 0.002s sys 0.036s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126328808105-19048-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126328808105-19048-0/krb5kdc/test-user.keytab.
Aug 07 18:17:36 dist-test-slave-gkw7 krb5kdc[12715](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126656, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:17:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:17:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.154.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.154.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.154.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.62.keytab.
Entry for principal kudu/127.18.154.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.62.keytab.
I20260807 18:17:36.379513 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126328808105-19048-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126328808105-19048-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126328808105-19048-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.154.62:34455
--webserver_interface=127.18.154.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.62.keytab
--principal=kudu/127.18.154.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.154.20:38269
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.154.62:34455
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126328808105-19048-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126328808105-19048-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126328808105-19048-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:17:36.726593 12735 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:17:36.726953 12735 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:17:36.727139 12735 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:17:36.734392 12735 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:17:36.734541 12735 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:17:36.734647 12735 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:17:36.734747 12735 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:17:36.734820 12735 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:17:36.744643 12735 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.154.20:38269
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126328808105-19048-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126328808105-19048-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.154.62:34455
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.62.keytab
--principal=kudu/127.18.154.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.154.62:34455
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.154.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126328808105-19048-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:17:36.746069 12735 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:17:36.747429 12735 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:17:36.759609 12752 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:17:36.759653 12748 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:17:36.759560 12744 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:17:36.763031 12735 server_base.cc:1034] running on GCE node
I20260807 18:17:36.763876 12735 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:17:36.765321 12735 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:17:36.766528 12735 hybrid_clock.cc:648] HybridClock initialized: now 1786126656766476 us; error 37 us; skew 500 ppm
I20260807 18:17:36.766903 12735 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:17:36 dist-test-slave-gkw7 krb5kdc[12715](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126656, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:17:36.773310 12735 init.cc:377] Logged in from keytab as kudu/127.18.154.62@KRBTEST.COM (short username kudu)
I20260807 18:17:36.775027 12735 webserver.cc:460] Webserver started at http://127.18.154.62:43669/ using document root <none> and password file <none>
I20260807 18:17:36.775775 12735 fs_manager.cc:359] Metadata directory not provided
I20260807 18:17:36.775933 12735 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:17:36.776253 12735 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:17:36.779327 12735 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126328808105-19048-0/minicluster-data/master-0/data/instance:
uuid: "7e43081825ea4f97a226f34b4acfac54"
format_stamp: "Formatted at 2026-08-07 18:17:36 on dist-test-slave-gkw7"
server_key: "e5a3d1c1fc28742b60e32c17dd270b01"
server_key_iv: "48b3c84942fe0678ee8e19368c5f33a6"
server_key_version: "encryptionkey@0"
I20260807 18:17:36.780108 12735 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126328808105-19048-0/minicluster-data/master-0/wal/instance:
uuid: "7e43081825ea4f97a226f34b4acfac54"
format_stamp: "Formatted at 2026-08-07 18:17:36 on dist-test-slave-gkw7"
server_key: "e5a3d1c1fc28742b60e32c17dd270b01"
server_key_iv: "48b3c84942fe0678ee8e19368c5f33a6"
server_key_version: "encryptionkey@0"
I20260807 18:17:36.785373 12735 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260807 18:17:36.788761 12764 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:17:36.790163 12735 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:17:36.790361 12735 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126328808105-19048-0/minicluster-data/master-0/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126328808105-19048-0/minicluster-data/master-0/wal
uuid: "7e43081825ea4f97a226f34b4acfac54"
format_stamp: "Formatted at 2026-08-07 18:17:36 on dist-test-slave-gkw7"
server_key: "e5a3d1c1fc28742b60e32c17dd270b01"
server_key_iv: "48b3c84942fe0678ee8e19368c5f33a6"
server_key_version: "encryptionkey@0"
I20260807 18:17:36.790561 12735 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126328808105-19048-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126328808105-19048-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126328808105-19048-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:17:36.818552 12735 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:17:36.822923 12735 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:17:36.823235 12735 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:17:36.843037 12735 rpc_server.cc:307] RPC server started. Bound to: 127.18.154.62:34455
I20260807 18:17:36.843051 12830 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.154.62:34455 every 8 connection(s)
I20260807 18:17:36.844532 12735 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
I20260807 18:17:36.848183 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 12735
I20260807 18:17:36.848415 19048 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126328808105-19048-0/minicluster-data/master-0/wal/instance
I20260807 18:17:36.848703 12832 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:17:36.848910 19048 external_mini_cluster.cc:1428] Setting key cf89fbebd6025e014ac9063df70d212b
I20260807 18:17:36.864609 12832 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e43081825ea4f97a226f34b4acfac54: Bootstrap starting.
Aug 07 18:17:36 dist-test-slave-gkw7 krb5kdc[12715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126656, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.154.62@KRBTEST.COM
I20260807 18:17:36.868495 12832 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7e43081825ea4f97a226f34b4acfac54: Neither blocks nor log segments found. Creating new log.
I20260807 18:17:36.869643 12832 log.cc:824] T 00000000000000000000000000000000 P 7e43081825ea4f97a226f34b4acfac54: Log is configured to *not* fsync() on all Append() calls
I20260807 18:17:36.872982 12832 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e43081825ea4f97a226f34b4acfac54: No bootstrap required, opened a new log
I20260807 18:17:36.874380 12835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:36.850882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58190 (local address 127.18.154.62:34455)
0807 18:17:36.852035 (+ 1153us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:36.852050 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:36.852093 (+ 43us) server_negotiation.cc:407] Connection header received
0807 18:17:36.853948 (+ 1855us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:36.853973 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:36.854390 (+ 417us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:36.854756 (+ 366us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:17:36.856779 (+ 2023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:36.858257 (+ 1478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:36.860503 (+ 2246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:36.861217 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:36.861358 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:36.866825 (+ 5467us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:17:36.866833 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:17:36.866849 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:17:36.866893 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:17:36.870207 (+ 3314us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:17:36.870890 (+ 683us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:17:36.870900 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:17:36.870908 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:17:36.871089 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:17:36.871579 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:17:36.871583 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:17:36.871585 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:17:36.872120 (+ 535us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:17:36.872333 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:36.872874 (+ 541us) server_negotiation.cc:299] Negotiation successful
0807 18:17:36.873319 (+ 445us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":880,"thread_start_us":543,"threads_started":1}
I20260807 18:17:36.880823 12832 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7e43081825ea4f97a226f34b4acfac54 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e43081825ea4f97a226f34b4acfac54" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 34455 } }
I20260807 18:17:36.881181 12832 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7e43081825ea4f97a226f34b4acfac54 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:17:36.881304 12832 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7e43081825ea4f97a226f34b4acfac54 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e43081825ea4f97a226f34b4acfac54, State: Initialized, Role: FOLLOWER
I20260807 18:17:36.881917 12832 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7e43081825ea4f97a226f34b4acfac54 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e43081825ea4f97a226f34b4acfac54" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 34455 } }
I20260807 18:17:36.882189 12832 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7e43081825ea4f97a226f34b4acfac54 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:17:36.882341 12832 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7e43081825ea4f97a226f34b4acfac54 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:17:36.882539 12832 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7e43081825ea4f97a226f34b4acfac54 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:17:36.884631 12832 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7e43081825ea4f97a226f34b4acfac54 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e43081825ea4f97a226f34b4acfac54" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 34455 } }
I20260807 18:17:36.885135 12832 leader_election.cc:302] T 00000000000000000000000000000000 P 7e43081825ea4f97a226f34b4acfac54 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7e43081825ea4f97a226f34b4acfac54; no voters:
I20260807 18:17:36.885871 12832 leader_election.cc:288] T 00000000000000000000000000000000 P 7e43081825ea4f97a226f34b4acfac54 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:17:36.886016 12837 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7e43081825ea4f97a226f34b4acfac54 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:17:36.886729 12837 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7e43081825ea4f97a226f34b4acfac54 [term 1 LEADER]: Becoming Leader. State: Replica: 7e43081825ea4f97a226f34b4acfac54, State: Running, Role: LEADER
I20260807 18:17:36.887244 12837 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7e43081825ea4f97a226f34b4acfac54 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e43081825ea4f97a226f34b4acfac54" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 34455 } }
I20260807 18:17:36.887960 12832 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7e43081825ea4f97a226f34b4acfac54 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:17:36.890288 12840 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e43081825ea4f97a226f34b4acfac54 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7e43081825ea4f97a226f34b4acfac54. Latest consensus state: current_term: 1 leader_uuid: "7e43081825ea4f97a226f34b4acfac54" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e43081825ea4f97a226f34b4acfac54" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 34455 } } }
I20260807 18:17:36.890743 12840 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e43081825ea4f97a226f34b4acfac54 [sys.catalog]: This master's current role is: LEADER
I20260807 18:17:36.893198 12847 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:17:36.893079 12838 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e43081825ea4f97a226f34b4acfac54 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7e43081825ea4f97a226f34b4acfac54" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e43081825ea4f97a226f34b4acfac54" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 34455 } } }
I20260807 18:17:36.893427 12838 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e43081825ea4f97a226f34b4acfac54 [sys.catalog]: This master's current role is: LEADER
I20260807 18:17:36.898873 12847 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:17:36.908341 12847 catalog_manager.cc:1347] Generated new cluster ID: 40bd57c490e146c6a77f30cd6ad9c451
I20260807 18:17:36.908445 12847 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:17:36.922688 12847 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:17:36.924573 12847 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:17:36.932050 12847 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7e43081825ea4f97a226f34b4acfac54: Generated new TSK 0
I20260807 18:17:36.932878 12847 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.154.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.154.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.154.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.1.keytab.
Entry for principal kudu/127.18.154.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.1.keytab.
I20260807 18:17:37.014606 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126328808105-19048-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126328808105-19048-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126328808105-19048-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126328808105-19048-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.154.1:0
--local_ip_for_outbound_sockets=127.18.154.1
--webserver_interface=127.18.154.1
--webserver_port=0
--tserver_master_addrs=127.18.154.62:34455
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.1.keytab
--principal=kudu/127.18.154.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.154.20:38269
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126328808105-19048-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126328808105-19048-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126328808105-19048-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:17:37.368669 12869 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:17:37.368949 12869 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:17:37.369042 12869 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:17:37.376705 12869 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:17:37.376816 12869 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:17:37.376935 12869 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.154.1
I20260807 18:17:37.386761 12869 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.154.20:38269
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126328808105-19048-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126328808105-19048-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.1.keytab
--principal=kudu/127.18.154.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.154.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126328808105-19048-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.154.1
--webserver_port=0
--tserver_master_addrs=127.18.154.62:34455
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.154.1
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126328808105-19048-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:17:37.388118 12869 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:17:37.389475 12869 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:17:37.403244 12875 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:17:37.403313 12876 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:17:37.403334 12880 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:17:37.404168 12869 server_base.cc:1034] running on GCE node
I20260807 18:17:37.404874 12869 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:17:37.405846 12869 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:17:37.407083 12869 hybrid_clock.cc:648] HybridClock initialized: now 1786126657407014 us; error 63 us; skew 500 ppm
I20260807 18:17:37.407460 12869 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:17:37 dist-test-slave-gkw7 krb5kdc[12715](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126657, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:17:37.410872 12869 init.cc:377] Logged in from keytab as kudu/127.18.154.1@KRBTEST.COM (short username kudu)
I20260807 18:17:37.412461 12869 webserver.cc:460] Webserver started at http://127.18.154.1:42063/ using document root <none> and password file <none>
I20260807 18:17:37.413173 12869 fs_manager.cc:359] Metadata directory not provided
I20260807 18:17:37.413331 12869 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:17:37.413745 12869 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:17:37.416694 12869 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126328808105-19048-0/minicluster-data/ts-0/data/instance:
uuid: "5dd823ded93f47f9bc48fb5795e45b60"
format_stamp: "Formatted at 2026-08-07 18:17:37 on dist-test-slave-gkw7"
server_key: "d8abf0916ec81b346a7146733819f65d"
server_key_iv: "b9497cbeb4960179bfa0dcf9f37384bc"
server_key_version: "encryptionkey@0"
I20260807 18:17:37.417641 12869 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126328808105-19048-0/minicluster-data/ts-0/wal/instance:
uuid: "5dd823ded93f47f9bc48fb5795e45b60"
format_stamp: "Formatted at 2026-08-07 18:17:37 on dist-test-slave-gkw7"
server_key: "d8abf0916ec81b346a7146733819f65d"
server_key_iv: "b9497cbeb4960179bfa0dcf9f37384bc"
server_key_version: "encryptionkey@0"
I20260807 18:17:37.423048 12869 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260807 18:17:37.426746 12891 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:17:37.428174 12869 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 18:17:37.428380 12869 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126328808105-19048-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126328808105-19048-0/minicluster-data/ts-0/wal
uuid: "5dd823ded93f47f9bc48fb5795e45b60"
format_stamp: "Formatted at 2026-08-07 18:17:37 on dist-test-slave-gkw7"
server_key: "d8abf0916ec81b346a7146733819f65d"
server_key_iv: "b9497cbeb4960179bfa0dcf9f37384bc"
server_key_version: "encryptionkey@0"
I20260807 18:17:37.428596 12869 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126328808105-19048-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126328808105-19048-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126328808105-19048-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:17:37.451009 12869 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:17:37.455675 12869 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:17:37.455983 12869 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:17:37.456975 12869 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:17:37.458400 12869 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:17:37.458505 12869 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:17:37.458652 12869 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:17:37.458720 12869 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:17:37.493353 12869 rpc_server.cc:307] RPC server started. Bound to: 127.18.154.1:38817
I20260807 18:17:37.493474 13042 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.154.1:38817 every 8 connection(s)
I20260807 18:17:37.495047 12869 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126328808105-19048-0/minicluster-data/ts-0/data/info.pb
I20260807 18:17:37.495452 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 12869
I20260807 18:17:37.495680 19048 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126328808105-19048-0/minicluster-data/ts-0/wal/instance
I20260807 18:17:37.496376 19048 external_mini_cluster.cc:1428] Setting key f281dabb44e2311e405b6c591233dc77
Aug 07 18:17:37 dist-test-slave-gkw7 krb5kdc[12715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126657, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.1@KRBTEST.COM for kudu/127.18.154.62@KRBTEST.COM
I20260807 18:17:37.520408 13047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:37.498170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.1:37237 (local address 127.18.154.62:34455)
0807 18:17:37.498988 (+ 818us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:37.498994 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:37.500013 (+ 1019us) server_negotiation.cc:407] Connection header received
0807 18:17:37.503058 (+ 3045us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:37.503062 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:37.503139 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:37.503239 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:17:37.505561 (+ 2322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:37.506606 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:37.509158 (+ 2552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:37.509839 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:37.509968 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:37.514471 (+ 4503us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:17:37.514476 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:17:37.514479 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:17:37.514515 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:17:37.517080 (+ 2565us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:17:37.518070 (+ 990us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:17:37.518074 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:17:37.518077 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:17:37.518192 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:17:37.518674 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:17:37.518677 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:17:37.518679 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:17:37.518936 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:17:37.519044 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:37.519965 (+ 921us) server_negotiation.cc:299] Negotiation successful
0807 18:17:37.520131 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":692,"thread_start_us":564,"threads_started":1}
I20260807 18:17:37.522102 13049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:37.498938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:34455 (local address 127.18.154.1:37237)
0807 18:17:37.499821 (+ 883us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:37.499886 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0807 18:17:37.502719 (+ 2833us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:17:37.503540 (+ 821us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:17:37.503553 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:17:37.504439 (+ 886us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:17:37.505330 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:37.505360 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:37.506773 (+ 1413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:37.506780 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:37.508999 (+ 2219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:37.509017 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:37.510077 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:37.510086 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:37.510435 (+ 349us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:37.511277 (+ 842us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:17:37.511316 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:17:37.514229 (+ 2913us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:17:37.517233 (+ 3004us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:17:37.517246 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:17:37.517269 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:17:37.517961 (+ 692us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:17:37.518300 (+ 339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:17:37.518307 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:17:37.518314 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:17:37.518565 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:17:37.519069 (+ 504us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:17:37.519080 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:17:37.519541 (+ 461us) client_negotiation.cc:769] Sending connection context
0807 18:17:37.519872 (+ 331us) client_negotiation.cc:240] Negotiation successful
0807 18:17:37.520469 (+ 597us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":555,"thread_start_us":315,"threads_started":1}
I20260807 18:17:37.523770 13043 heartbeater.cc:347] Connected to a master server at 127.18.154.62:34455
I20260807 18:17:37.524106 13043 heartbeater.cc:464] Registering TS with master...
I20260807 18:17:37.525080 13043 heartbeater.cc:511] Master 127.18.154.62:34455 requested a full tablet report, sending...
I20260807 18:17:37.527540 12783 ts_manager.cc:194] Registered new tserver with Master: 5dd823ded93f47f9bc48fb5795e45b60 (127.18.154.1:38817)
I20260807 18:17:37.529047 12783 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.154.1@KRBTEST.COM'} at 127.18.154.1:37237
WARNING: no policy specified for kudu/127.18.154.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.154.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.154.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.2.keytab.
Entry for principal kudu/127.18.154.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.2.keytab.
I20260807 18:17:37.570900 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126328808105-19048-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126328808105-19048-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126328808105-19048-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126328808105-19048-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.154.2:0
--local_ip_for_outbound_sockets=127.18.154.2
--webserver_interface=127.18.154.2
--webserver_port=0
--tserver_master_addrs=127.18.154.62:34455
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.2.keytab
--principal=kudu/127.18.154.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.154.20:38269
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126328808105-19048-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126328808105-19048-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126328808105-19048-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:17:37.927526 13055 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:17:37.927812 13055 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:17:37.927914 13055 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:17:37.935273 13055 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:17:37.935389 13055 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:17:37.935513 13055 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.154.2
I20260807 18:17:37.945287 13055 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.154.20:38269
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126328808105-19048-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126328808105-19048-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.2.keytab
--principal=kudu/127.18.154.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.154.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126328808105-19048-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.154.2
--webserver_port=0
--tserver_master_addrs=127.18.154.62:34455
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.154.2
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126328808105-19048-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:17:37.946645 13055 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:17:37.947912 13055 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:17:37.961593 13066 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:17:37.961663 13064 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:17:37.961689 13063 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:17:37.962558 13055 server_base.cc:1034] running on GCE node
I20260807 18:17:37.963212 13055 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:17:37.964207 13055 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:17:37.965444 13055 hybrid_clock.cc:648] HybridClock initialized: now 1786126657965396 us; error 56 us; skew 500 ppm
I20260807 18:17:37.965801 13055 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:17:37 dist-test-slave-gkw7 krb5kdc[12715](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126657, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:17:37.969231 13055 init.cc:377] Logged in from keytab as kudu/127.18.154.2@KRBTEST.COM (short username kudu)
I20260807 18:17:37.970777 13055 webserver.cc:460] Webserver started at http://127.18.154.2:35465/ using document root <none> and password file <none>
I20260807 18:17:37.971535 13055 fs_manager.cc:359] Metadata directory not provided
I20260807 18:17:37.971652 13055 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:17:37.971936 13055 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:17:37.975100 13055 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126328808105-19048-0/minicluster-data/ts-1/data/instance:
uuid: "2e6115fe9cc74b2dad3b4948a436e22a"
format_stamp: "Formatted at 2026-08-07 18:17:37 on dist-test-slave-gkw7"
server_key: "000666b56595353b23486b98f7456267"
server_key_iv: "d1215f648e8e1af39e4e40f77ba3d7ab"
server_key_version: "encryptionkey@0"
I20260807 18:17:37.975888 13055 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126328808105-19048-0/minicluster-data/ts-1/wal/instance:
uuid: "2e6115fe9cc74b2dad3b4948a436e22a"
format_stamp: "Formatted at 2026-08-07 18:17:37 on dist-test-slave-gkw7"
server_key: "000666b56595353b23486b98f7456267"
server_key_iv: "d1215f648e8e1af39e4e40f77ba3d7ab"
server_key_version: "encryptionkey@0"
I20260807 18:17:37.981513 13055 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260807 18:17:37.985054 13074 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:17:37.986477 13055 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260807 18:17:37.986660 13055 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126328808105-19048-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126328808105-19048-0/minicluster-data/ts-1/wal
uuid: "2e6115fe9cc74b2dad3b4948a436e22a"
format_stamp: "Formatted at 2026-08-07 18:17:37 on dist-test-slave-gkw7"
server_key: "000666b56595353b23486b98f7456267"
server_key_iv: "d1215f648e8e1af39e4e40f77ba3d7ab"
server_key_version: "encryptionkey@0"
I20260807 18:17:37.986864 13055 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126328808105-19048-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126328808105-19048-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126328808105-19048-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:17:38.023454 13055 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:17:38.028008 13055 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:17:38.028321 13055 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:17:38.029327 13055 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:17:38.030792 13055 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:17:38.030870 13055 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:17:38.031051 13055 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:17:38.031104 13055 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:17:38.065907 13055 rpc_server.cc:307] RPC server started. Bound to: 127.18.154.2:44357
I20260807 18:17:38.066001 13227 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.154.2:44357 every 8 connection(s)
I20260807 18:17:38.067374 13055 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126328808105-19048-0/minicluster-data/ts-1/data/info.pb
I20260807 18:17:38.073025 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 13055
I20260807 18:17:38.073216 19048 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126328808105-19048-0/minicluster-data/ts-1/wal/instance
I20260807 18:17:38.073724 19048 external_mini_cluster.cc:1428] Setting key 2a2c4c9f4fbf1f11096241b2dd6f484d
Aug 07 18:17:38 dist-test-slave-gkw7 krb5kdc[12715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126657, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.2@KRBTEST.COM for kudu/127.18.154.62@KRBTEST.COM
I20260807 18:17:38.089936 13231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:38.070150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.2:49805 (local address 127.18.154.62:34455)
0807 18:17:38.070644 (+ 494us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:38.070649 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:38.071776 (+ 1127us) server_negotiation.cc:407] Connection header received
0807 18:17:38.074024 (+ 2248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:38.074030 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:38.074152 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:38.074290 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:17:38.076014 (+ 1724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:38.076845 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:38.078746 (+ 1901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:38.079238 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:38.079390 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:38.083325 (+ 3935us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:17:38.083331 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:17:38.083335 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:17:38.083371 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:17:38.086377 (+ 3006us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:17:38.087211 (+ 834us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:17:38.087216 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:17:38.087219 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:17:38.087363 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:17:38.087850 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:17:38.087854 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:17:38.087857 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:17:38.088243 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:17:38.088412 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:38.089225 (+ 813us) server_negotiation.cc:299] Negotiation successful
0807 18:17:38.089478 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":370,"thread_start_us":234,"threads_started":1}
I20260807 18:17:38.091253 13233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:38.070753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:34455 (local address 127.18.154.2:49805)
0807 18:17:38.071586 (+ 833us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:38.071634 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 18:17:38.073676 (+ 2042us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:17:38.074575 (+ 899us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:17:38.074585 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:17:38.075187 (+ 602us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:17:38.075815 (+ 628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:38.075835 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:38.077008 (+ 1173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:38.077020 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:38.078586 (+ 1566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:38.078600 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:38.079495 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:38.079504 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:38.079788 (+ 284us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:38.080550 (+ 762us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:17:38.080584 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:17:38.083077 (+ 2493us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:17:38.086553 (+ 3476us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:17:38.086574 (+ 21us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:17:38.086593 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:17:38.087068 (+ 475us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:17:38.087484 (+ 416us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:17:38.087489 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:17:38.087494 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:17:38.087715 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:17:38.088385 (+ 670us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:17:38.088393 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:17:38.088820 (+ 427us) client_negotiation.cc:769] Sending connection context
0807 18:17:38.089106 (+ 286us) client_negotiation.cc:240] Negotiation successful
0807 18:17:38.089753 (+ 647us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":578,"thread_start_us":344,"threads_started":1}
I20260807 18:17:38.092986 13229 heartbeater.cc:347] Connected to a master server at 127.18.154.62:34455
I20260807 18:17:38.093339 13229 heartbeater.cc:464] Registering TS with master...
I20260807 18:17:38.094252 13229 heartbeater.cc:511] Master 127.18.154.62:34455 requested a full tablet report, sending...
I20260807 18:17:38.096010 12783 ts_manager.cc:194] Registered new tserver with Master: 2e6115fe9cc74b2dad3b4948a436e22a (127.18.154.2:44357)
I20260807 18:17:38.096879 12783 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.154.2@KRBTEST.COM'} at 127.18.154.2:49805
WARNING: no policy specified for kudu/127.18.154.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.154.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.154.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.3.keytab.
Entry for principal kudu/127.18.154.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.3.keytab.
I20260807 18:17:38.144711 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126328808105-19048-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126328808105-19048-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126328808105-19048-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126328808105-19048-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.154.3:0
--local_ip_for_outbound_sockets=127.18.154.3
--webserver_interface=127.18.154.3
--webserver_port=0
--tserver_master_addrs=127.18.154.62:34455
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.3.keytab
--principal=kudu/127.18.154.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.154.20:38269
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126328808105-19048-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126328808105-19048-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126328808105-19048-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:17:38.493522 13236 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:17:38.493897 13236 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:17:38.494068 13236 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:17:38.502077 13236 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:17:38.502226 13236 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:17:38.502414 13236 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.154.3
I20260807 18:17:38.512507 13236 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.154.20:38269
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126328808105-19048-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126328808105-19048-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.3.keytab
--principal=kudu/127.18.154.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.154.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126328808105-19048-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.154.3
--webserver_port=0
--tserver_master_addrs=127.18.154.62:34455
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.154.3
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126328808105-19048-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:17:38.513924 13236 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:17:38.515381 13236 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:17:38.528843 13243 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:17:38.528820 13244 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:17:38.528865 13246 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:17:38.529603 13236 server_base.cc:1034] running on GCE node
I20260807 18:17:38.530285 13236 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:17:38.531251 13236 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:17:38.532490 13236 hybrid_clock.cc:648] HybridClock initialized: now 1786126658532436 us; error 61 us; skew 500 ppm
I20260807 18:17:38.532838 13236 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 18:17:38.532915 13043 heartbeater.cc:503] Master 127.18.154.62:34455 was elected leader, sending a full tablet report...
Aug 07 18:17:38 dist-test-slave-gkw7 krb5kdc[12715](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126658, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:17:38.536383 13236 init.cc:377] Logged in from keytab as kudu/127.18.154.3@KRBTEST.COM (short username kudu)
I20260807 18:17:38.537958 13236 webserver.cc:460] Webserver started at http://127.18.154.3:38603/ using document root <none> and password file <none>
I20260807 18:17:38.538669 13236 fs_manager.cc:359] Metadata directory not provided
I20260807 18:17:38.538823 13236 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:17:38.539144 13236 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:17:38.542168 13236 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126328808105-19048-0/minicluster-data/ts-2/data/instance:
uuid: "3f33588d4ac44320875c77d559c352a6"
format_stamp: "Formatted at 2026-08-07 18:17:38 on dist-test-slave-gkw7"
server_key: "b48857229f939cf052647bf862b0e778"
server_key_iv: "8fc213e1b33a9ab3f742d70b03273e14"
server_key_version: "encryptionkey@0"
I20260807 18:17:38.542968 13236 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126328808105-19048-0/minicluster-data/ts-2/wal/instance:
uuid: "3f33588d4ac44320875c77d559c352a6"
format_stamp: "Formatted at 2026-08-07 18:17:38 on dist-test-slave-gkw7"
server_key: "b48857229f939cf052647bf862b0e778"
server_key_iv: "8fc213e1b33a9ab3f742d70b03273e14"
server_key_version: "encryptionkey@0"
I20260807 18:17:38.548336 13236 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260807 18:17:38.551860 13255 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:17:38.553160 13236 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260807 18:17:38.553341 13236 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126328808105-19048-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126328808105-19048-0/minicluster-data/ts-2/wal
uuid: "3f33588d4ac44320875c77d559c352a6"
format_stamp: "Formatted at 2026-08-07 18:17:38 on dist-test-slave-gkw7"
server_key: "b48857229f939cf052647bf862b0e778"
server_key_iv: "8fc213e1b33a9ab3f742d70b03273e14"
server_key_version: "encryptionkey@0"
I20260807 18:17:38.553594 13236 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126328808105-19048-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126328808105-19048-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126328808105-19048-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:17:38.591585 13236 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:17:38.596405 13236 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:17:38.596717 13236 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:17:38.597761 13236 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:17:38.599109 13236 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:17:38.599215 13236 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:17:38.599368 13236 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:17:38.599438 13236 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:17:38.635588 13236 rpc_server.cc:307] RPC server started. Bound to: 127.18.154.3:41901
I20260807 18:17:38.635679 13410 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.154.3:41901 every 8 connection(s)
I20260807 18:17:38.636981 13236 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126328808105-19048-0/minicluster-data/ts-2/data/info.pb
I20260807 18:17:38.637398 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 13236
I20260807 18:17:38.637652 19048 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126328808105-19048-0/minicluster-data/ts-2/wal/instance
I20260807 18:17:38.638309 19048 external_mini_cluster.cc:1428] Setting key 9ea27d08b5b9b6da784e51d2489acd52
Aug 07 18:17:38 dist-test-slave-gkw7 krb5kdc[12715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126658, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.3@KRBTEST.COM for kudu/127.18.154.62@KRBTEST.COM
I20260807 18:17:38.657852 13415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:38.639808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.3:47813 (local address 127.18.154.62:34455)
0807 18:17:38.640345 (+ 537us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:38.640352 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:38.641461 (+ 1109us) server_negotiation.cc:407] Connection header received
0807 18:17:38.643359 (+ 1898us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:38.643363 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:38.643445 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:38.643536 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:17:38.645021 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:38.645890 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:38.647390 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:38.647788 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:38.647882 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:38.651313 (+ 3431us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:17:38.651318 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:17:38.651321 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:17:38.651355 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:17:38.654645 (+ 3290us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:17:38.655397 (+ 752us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:17:38.655404 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:17:38.655408 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:17:38.655541 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:17:38.655969 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:17:38.655975 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:17:38.655979 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:17:38.656256 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:17:38.656418 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:38.657139 (+ 721us) server_negotiation.cc:299] Negotiation successful
0807 18:17:38.657391 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":404,"thread_start_us":266,"threads_started":1}
I20260807 18:17:38.658854 13416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:38.640477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:34455 (local address 127.18.154.3:47813)
0807 18:17:38.641303 (+ 826us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:38.641360 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0807 18:17:38.643113 (+ 1753us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:17:38.643748 (+ 635us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:17:38.643755 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:17:38.644301 (+ 546us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:17:38.644877 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:38.644894 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:38.646025 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:38.646029 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:38.647263 (+ 1234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:38.647275 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:38.647968 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:38.647972 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:38.648170 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:38.648813 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:17:38.648835 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:17:38.651086 (+ 2251us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:17:38.654814 (+ 3728us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:17:38.654824 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:17:38.654846 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:17:38.655279 (+ 433us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:17:38.655653 (+ 374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:17:38.655658 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:17:38.655662 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:17:38.655857 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:17:38.656390 (+ 533us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:17:38.656399 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:17:38.656786 (+ 387us) client_negotiation.cc:769] Sending connection context
0807 18:17:38.657060 (+ 274us) client_negotiation.cc:240] Negotiation successful
0807 18:17:38.657659 (+ 599us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":547,"thread_start_us":356,"threads_started":1}
I20260807 18:17:38.660212 13411 heartbeater.cc:347] Connected to a master server at 127.18.154.62:34455
I20260807 18:17:38.660502 13411 heartbeater.cc:464] Registering TS with master...
I20260807 18:17:38.661190 13411 heartbeater.cc:511] Master 127.18.154.62:34455 requested a full tablet report, sending...
I20260807 18:17:38.662652 12783 ts_manager.cc:194] Registered new tserver with Master: 3f33588d4ac44320875c77d559c352a6 (127.18.154.3:41901)
I20260807 18:17:38.663462 12783 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.154.3@KRBTEST.COM'} at 127.18.154.3:47813
I20260807 18:17:38.666345 19048 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 18:17:38.683957 13415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:38.670319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58200 (local address 127.18.154.62:34455)
0807 18:17:38.670506 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:38.670514 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:38.670872 (+ 358us) server_negotiation.cc:407] Connection header received
0807 18:17:38.671068 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:38.671074 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:38.671143 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:38.671245 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:17:38.672635 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:38.673376 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:38.674834 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:38.675264 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:38.675397 (+ 133us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:38.678552 (+ 3155us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:17:38.678558 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:17:38.678562 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:17:38.678593 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:17:38.681047 (+ 2454us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:17:38.681852 (+ 805us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:17:38.681857 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:17:38.681865 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:17:38.681994 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:17:38.682497 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:17:38.682501 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:17:38.682503 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:17:38.682812 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:17:38.683082 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:38.683476 (+ 394us) server_negotiation.cc:299] Negotiation successful
0807 18:17:38.683663 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260807 18:17:38.687768 12783 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58200:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 18:17:38.694888 12783 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:17:38.719930 13434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:38.707607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41792 (local address 127.18.154.2:44357)
0807 18:17:38.709775 (+ 2168us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:38.709789 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:38.710392 (+ 603us) server_negotiation.cc:407] Connection header received
0807 18:17:38.710700 (+ 308us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:38.710708 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:38.710958 (+ 250us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:38.711133 (+ 175us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:17:38.712779 (+ 1646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:38.714315 (+ 1536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:38.716855 (+ 2540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:38.718299 (+ 1444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:38.718448 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:38.718664 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:38.719388 (+ 724us) server_negotiation.cc:299] Negotiation successful
0807 18:17:38.719578 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1944,"thread_start_us":1775,"threads_started":1}
I20260807 18:17:38.720008 13432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:38.706906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.1:38817 (local address 127.0.0.1:53176)
0807 18:17:38.708631 (+ 1725us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:38.708678 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 18:17:38.708848 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:17:38.711204 (+ 2356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:17:38.711212 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:17:38.711247 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:17:38.711814 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:38.711833 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:38.714425 (+ 2592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:38.714433 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:38.717082 (+ 2649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:38.717098 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:38.719001 (+ 1903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:38.719007 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:38.719167 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:38.719198 (+ 31us) client_negotiation.cc:769] Sending connection context
0807 18:17:38.719294 (+ 96us) client_negotiation.cc:240] Negotiation successful
0807 18:17:38.719492 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1485,"mutex_wait_us":113,"thread_start_us":5479,"threads_started":1}
I20260807 18:17:38.720520 13433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:38.707266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53176 (local address 127.18.154.1:38817)
0807 18:17:38.710617 (+ 3351us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:38.710626 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:38.710651 (+ 25us) server_negotiation.cc:407] Connection header received
0807 18:17:38.710750 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:38.710756 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:38.710957 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:38.711942 (+ 985us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:17:38.712830 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:38.714264 (+ 1434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:38.717266 (+ 3002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:38.718746 (+ 1480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:38.719675 (+ 929us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:38.719915 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:38.720006 (+ 91us) server_negotiation.cc:299] Negotiation successful
0807 18:17:38.720173 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3157,"thread_start_us":4308,"threads_started":1}
I20260807 18:17:38.720707 13435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:38.707097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.2:44357 (local address 127.0.0.1:41792)
0807 18:17:38.710269 (+ 3172us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:38.710289 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 18:17:38.710503 (+ 214us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:17:38.711450 (+ 947us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:17:38.711454 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:17:38.711475 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:17:38.712579 (+ 1104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:38.712593 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:38.714499 (+ 1906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:38.714505 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:38.716690 (+ 2185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:38.716710 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:38.718841 (+ 2131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:38.718848 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:38.719096 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:38.719159 (+ 63us) client_negotiation.cc:769] Sending connection context
0807 18:17:38.720184 (+ 1025us) client_negotiation.cc:240] Negotiation successful
0807 18:17:38.720319 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3021,"mutex_wait_us":105,"thread_start_us":1161,"threads_started":1}
I20260807 18:17:38.721004 13436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:38.709418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47690 (local address 127.18.154.3:41901)
0807 18:17:38.712409 (+ 2991us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:38.712418 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:38.712819 (+ 401us) server_negotiation.cc:407] Connection header received
0807 18:17:38.713005 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:38.713013 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:38.713253 (+ 240us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:38.713448 (+ 195us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:17:38.714038 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:38.715659 (+ 1621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:38.717931 (+ 2272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:38.719139 (+ 1208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:38.719284 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:38.719529 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:38.720451 (+ 922us) server_negotiation.cc:299] Negotiation successful
0807 18:17:38.720619 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2778,"thread_start_us":2604,"threads_started":1}
I20260807 18:17:38.721448 13428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:38.709060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.3:41901 (local address 127.0.0.1:47690)
0807 18:17:38.712706 (+ 3646us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:38.712725 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 18:17:38.712840 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:17:38.713460 (+ 620us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:17:38.713465 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:17:38.713491 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:17:38.713863 (+ 372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:38.713874 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:38.715813 (+ 1939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:38.715818 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:38.717775 (+ 1957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:38.717786 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:38.719686 (+ 1900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:38.719697 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:38.719877 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:38.719920 (+ 43us) client_negotiation.cc:769] Sending connection context
0807 18:17:38.720973 (+ 1053us) client_negotiation.cc:240] Negotiation successful
0807 18:17:38.721132 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3511,"thread_start_us":913,"threads_started":1}
I20260807 18:17:38.724589 13139 tablet_service.cc:1467] Processing CreateTablet for tablet 86979833ec734d95bacb54c987af1e0a (DEFAULT_TABLE table=test-table [id=edc6a82a4f1040a1a2eb9e0043a5c24c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:17:38.724577 12952 tablet_service.cc:1467] Processing CreateTablet for tablet 86979833ec734d95bacb54c987af1e0a (DEFAULT_TABLE table=test-table [id=edc6a82a4f1040a1a2eb9e0043a5c24c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:17:38.725955 13139 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 86979833ec734d95bacb54c987af1e0a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:17:38.726117 12952 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 86979833ec734d95bacb54c987af1e0a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:17:38.725632 13327 tablet_service.cc:1467] Processing CreateTablet for tablet 86979833ec734d95bacb54c987af1e0a (DEFAULT_TABLE table=test-table [id=edc6a82a4f1040a1a2eb9e0043a5c24c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:17:38.726979 13327 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 86979833ec734d95bacb54c987af1e0a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:17:38.738976 13438 tablet_bootstrap.cc:492] T 86979833ec734d95bacb54c987af1e0a P 2e6115fe9cc74b2dad3b4948a436e22a: Bootstrap starting.
I20260807 18:17:38.741032 13439 tablet_bootstrap.cc:492] T 86979833ec734d95bacb54c987af1e0a P 5dd823ded93f47f9bc48fb5795e45b60: Bootstrap starting.
I20260807 18:17:38.741899 13440 tablet_bootstrap.cc:492] T 86979833ec734d95bacb54c987af1e0a P 3f33588d4ac44320875c77d559c352a6: Bootstrap starting.
I20260807 18:17:38.742954 13438 tablet_bootstrap.cc:654] T 86979833ec734d95bacb54c987af1e0a P 2e6115fe9cc74b2dad3b4948a436e22a: Neither blocks nor log segments found. Creating new log.
I20260807 18:17:38.744318 13438 log.cc:824] T 86979833ec734d95bacb54c987af1e0a P 2e6115fe9cc74b2dad3b4948a436e22a: Log is configured to *not* fsync() on all Append() calls
I20260807 18:17:38.744742 13439 tablet_bootstrap.cc:654] T 86979833ec734d95bacb54c987af1e0a P 5dd823ded93f47f9bc48fb5795e45b60: Neither blocks nor log segments found. Creating new log.
I20260807 18:17:38.745617 13440 tablet_bootstrap.cc:654] T 86979833ec734d95bacb54c987af1e0a P 3f33588d4ac44320875c77d559c352a6: Neither blocks nor log segments found. Creating new log.
I20260807 18:17:38.745826 13439 log.cc:824] T 86979833ec734d95bacb54c987af1e0a P 5dd823ded93f47f9bc48fb5795e45b60: Log is configured to *not* fsync() on all Append() calls
I20260807 18:17:38.746989 13440 log.cc:824] T 86979833ec734d95bacb54c987af1e0a P 3f33588d4ac44320875c77d559c352a6: Log is configured to *not* fsync() on all Append() calls
I20260807 18:17:38.748775 13438 tablet_bootstrap.cc:492] T 86979833ec734d95bacb54c987af1e0a P 2e6115fe9cc74b2dad3b4948a436e22a: No bootstrap required, opened a new log
I20260807 18:17:38.749048 13438 ts_tablet_manager.cc:1397] T 86979833ec734d95bacb54c987af1e0a P 2e6115fe9cc74b2dad3b4948a436e22a: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.004s
I20260807 18:17:38.749680 13439 tablet_bootstrap.cc:492] T 86979833ec734d95bacb54c987af1e0a P 5dd823ded93f47f9bc48fb5795e45b60: No bootstrap required, opened a new log
I20260807 18:17:38.749969 13439 ts_tablet_manager.cc:1397] T 86979833ec734d95bacb54c987af1e0a P 5dd823ded93f47f9bc48fb5795e45b60: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.005s
I20260807 18:17:38.750217 13440 tablet_bootstrap.cc:492] T 86979833ec734d95bacb54c987af1e0a P 3f33588d4ac44320875c77d559c352a6: No bootstrap required, opened a new log
I20260807 18:17:38.750486 13440 ts_tablet_manager.cc:1397] T 86979833ec734d95bacb54c987af1e0a P 3f33588d4ac44320875c77d559c352a6: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.006s
I20260807 18:17:38.756285 13438 raft_consensus.cc:348] T 86979833ec734d95bacb54c987af1e0a P 2e6115fe9cc74b2dad3b4948a436e22a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dd823ded93f47f9bc48fb5795e45b60" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 38817 } } peers { permanent_uuid: "2e6115fe9cc74b2dad3b4948a436e22a" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 44357 } } peers { permanent_uuid: "3f33588d4ac44320875c77d559c352a6" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 41901 } }
I20260807 18:17:38.757558 13438 raft_consensus.cc:374] T 86979833ec734d95bacb54c987af1e0a P 2e6115fe9cc74b2dad3b4948a436e22a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:17:38.757694 13438 raft_consensus.cc:729] T 86979833ec734d95bacb54c987af1e0a P 2e6115fe9cc74b2dad3b4948a436e22a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e6115fe9cc74b2dad3b4948a436e22a, State: Initialized, Role: FOLLOWER
I20260807 18:17:38.758406 13438 consensus_queue.cc:260] T 86979833ec734d95bacb54c987af1e0a P 2e6115fe9cc74b2dad3b4948a436e22a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dd823ded93f47f9bc48fb5795e45b60" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 38817 } } peers { permanent_uuid: "2e6115fe9cc74b2dad3b4948a436e22a" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 44357 } } peers { permanent_uuid: "3f33588d4ac44320875c77d559c352a6" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 41901 } }
I20260807 18:17:38.759658 13229 heartbeater.cc:503] Master 127.18.154.62:34455 was elected leader, sending a full tablet report...
I20260807 18:17:38.760438 13438 ts_tablet_manager.cc:1428] T 86979833ec734d95bacb54c987af1e0a P 2e6115fe9cc74b2dad3b4948a436e22a: Time spent starting tablet: real 0.011s user 0.004s sys 0.008s
I20260807 18:17:38.761062 13439 raft_consensus.cc:348] T 86979833ec734d95bacb54c987af1e0a P 5dd823ded93f47f9bc48fb5795e45b60 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dd823ded93f47f9bc48fb5795e45b60" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 38817 } } peers { permanent_uuid: "2e6115fe9cc74b2dad3b4948a436e22a" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 44357 } } peers { permanent_uuid: "3f33588d4ac44320875c77d559c352a6" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 41901 } }
I20260807 18:17:38.761269 13440 raft_consensus.cc:348] T 86979833ec734d95bacb54c987af1e0a P 3f33588d4ac44320875c77d559c352a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dd823ded93f47f9bc48fb5795e45b60" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 38817 } } peers { permanent_uuid: "2e6115fe9cc74b2dad3b4948a436e22a" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 44357 } } peers { permanent_uuid: "3f33588d4ac44320875c77d559c352a6" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 41901 } }
I20260807 18:17:38.761670 13439 raft_consensus.cc:374] T 86979833ec734d95bacb54c987af1e0a P 5dd823ded93f47f9bc48fb5795e45b60 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:17:38.761716 13440 raft_consensus.cc:374] T 86979833ec734d95bacb54c987af1e0a P 3f33588d4ac44320875c77d559c352a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:17:38.761818 13439 raft_consensus.cc:729] T 86979833ec734d95bacb54c987af1e0a P 5dd823ded93f47f9bc48fb5795e45b60 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5dd823ded93f47f9bc48fb5795e45b60, State: Initialized, Role: FOLLOWER
I20260807 18:17:38.761871 13440 raft_consensus.cc:729] T 86979833ec734d95bacb54c987af1e0a P 3f33588d4ac44320875c77d559c352a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f33588d4ac44320875c77d559c352a6, State: Initialized, Role: FOLLOWER
I20260807 18:17:38.762529 13440 consensus_queue.cc:260] T 86979833ec734d95bacb54c987af1e0a P 3f33588d4ac44320875c77d559c352a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dd823ded93f47f9bc48fb5795e45b60" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 38817 } } peers { permanent_uuid: "2e6115fe9cc74b2dad3b4948a436e22a" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 44357 } } peers { permanent_uuid: "3f33588d4ac44320875c77d559c352a6" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 41901 } }
I20260807 18:17:38.762502 13439 consensus_queue.cc:260] T 86979833ec734d95bacb54c987af1e0a P 5dd823ded93f47f9bc48fb5795e45b60 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dd823ded93f47f9bc48fb5795e45b60" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 38817 } } peers { permanent_uuid: "2e6115fe9cc74b2dad3b4948a436e22a" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 44357 } } peers { permanent_uuid: "3f33588d4ac44320875c77d559c352a6" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 41901 } }
I20260807 18:17:38.764448 13440 ts_tablet_manager.cc:1428] T 86979833ec734d95bacb54c987af1e0a P 3f33588d4ac44320875c77d559c352a6: Time spent starting tablet: real 0.014s user 0.009s sys 0.004s
I20260807 18:17:38.764717 13439 ts_tablet_manager.cc:1428] T 86979833ec734d95bacb54c987af1e0a P 5dd823ded93f47f9bc48fb5795e45b60: Time spent starting tablet: real 0.015s user 0.010s sys 0.003s
I20260807 18:17:38.765058 13411 heartbeater.cc:503] Master 127.18.154.62:34455 was elected leader, sending a full tablet report...
W20260807 18:17:38.819599 13230 tablet.cc:2369] T 86979833ec734d95bacb54c987af1e0a P 2e6115fe9cc74b2dad3b4948a436e22a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:17:38.843986 13447 raft_consensus.cc:482] T 86979833ec734d95bacb54c987af1e0a P 2e6115fe9cc74b2dad3b4948a436e22a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:17:38.844369 13447 raft_consensus.cc:504] T 86979833ec734d95bacb54c987af1e0a P 2e6115fe9cc74b2dad3b4948a436e22a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dd823ded93f47f9bc48fb5795e45b60" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 38817 } } peers { permanent_uuid: "2e6115fe9cc74b2dad3b4948a436e22a" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 44357 } } peers { permanent_uuid: "3f33588d4ac44320875c77d559c352a6" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 41901 } }
I20260807 18:17:38.846307 13447 leader_election.cc:288] T 86979833ec734d95bacb54c987af1e0a P 2e6115fe9cc74b2dad3b4948a436e22a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5dd823ded93f47f9bc48fb5795e45b60 (127.18.154.1:38817), 3f33588d4ac44320875c77d559c352a6 (127.18.154.3:41901)
I20260807 18:17:38.854275 13433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:38.846957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.2:37851 (local address 127.18.154.1:38817)
0807 18:17:38.847221 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:38.847227 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:38.848168 (+ 941us) server_negotiation.cc:407] Connection header received
0807 18:17:38.848400 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:38.848408 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:38.848488 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:38.848585 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:17:38.849320 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:38.850434 (+ 1114us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:38.852390 (+ 1956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:38.853296 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:38.853457 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:38.853569 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:38.853883 (+ 314us) server_negotiation.cc:299] Negotiation successful
0807 18:17:38.854018 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106}
I20260807 18:17:38.854357 13450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:38.846668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.1:38817 (local address 127.18.154.2:37851)
0807 18:17:38.848076 (+ 1408us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:38.848093 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 18:17:38.848258 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:17:38.848618 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:17:38.848622 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:17:38.848666 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:17:38.849139 (+ 473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:38.849155 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:38.850566 (+ 1411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:38.850573 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:38.852237 (+ 1664us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:38.852250 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:38.853507 (+ 1257us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:38.853513 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:38.853708 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:38.853745 (+ 37us) client_negotiation.cc:769] Sending connection context
0807 18:17:38.853839 (+ 94us) client_negotiation.cc:240] Negotiation successful
0807 18:17:38.853966 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1295,"mutex_wait_us":79,"thread_start_us":732,"threads_started":1}
I20260807 18:17:38.855515 13436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:38.847096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.2:41919 (local address 127.18.154.3:41901)
0807 18:17:38.847291 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:38.847299 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:38.847905 (+ 606us) server_negotiation.cc:407] Connection header received
0807 18:17:38.848241 (+ 336us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:38.848248 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:38.848327 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:38.848428 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:17:38.849142 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:38.850337 (+ 1195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:38.853063 (+ 2726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:38.854181 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:38.854340 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:38.854430 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:38.855133 (+ 703us) server_negotiation.cc:299] Negotiation successful
0807 18:17:38.855262 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260807 18:17:38.855567 13451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:38.846612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.3:41901 (local address 127.18.154.2:41919)
0807 18:17:38.847769 (+ 1157us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:38.847795 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0807 18:17:38.848033 (+ 238us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:17:38.848448 (+ 415us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:17:38.848452 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:17:38.848476 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:17:38.848990 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:38.849001 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:38.850481 (+ 1480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:38.850486 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:38.852891 (+ 2405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:38.852906 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:38.854595 (+ 1689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:38.854601 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:38.854775 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:38.854808 (+ 33us) client_negotiation.cc:769] Sending connection context
0807 18:17:38.854921 (+ 113us) client_negotiation.cc:240] Negotiation successful
0807 18:17:38.855176 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":988,"mutex_wait_us":169,"spinlock_wait_cycles":28544,"thread_start_us":1143,"threads_started":1}
I20260807 18:17:38.855669 12979 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86979833ec734d95bacb54c987af1e0a" candidate_uuid: "2e6115fe9cc74b2dad3b4948a436e22a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5dd823ded93f47f9bc48fb5795e45b60" is_pre_election: true
I20260807 18:17:38.856287 12979 raft_consensus.cc:2454] T 86979833ec734d95bacb54c987af1e0a P 5dd823ded93f47f9bc48fb5795e45b60 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2e6115fe9cc74b2dad3b4948a436e22a in term 0.
I20260807 18:17:38.856689 13350 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86979833ec734d95bacb54c987af1e0a" candidate_uuid: "2e6115fe9cc74b2dad3b4948a436e22a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f33588d4ac44320875c77d559c352a6" is_pre_election: true
I20260807 18:17:38.856988 13079 leader_election.cc:302] T 86979833ec734d95bacb54c987af1e0a P 2e6115fe9cc74b2dad3b4948a436e22a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2e6115fe9cc74b2dad3b4948a436e22a, 5dd823ded93f47f9bc48fb5795e45b60; no voters:
I20260807 18:17:38.857240 13350 raft_consensus.cc:2454] T 86979833ec734d95bacb54c987af1e0a P 3f33588d4ac44320875c77d559c352a6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2e6115fe9cc74b2dad3b4948a436e22a in term 0.
I20260807 18:17:38.857400 13447 raft_consensus.cc:2781] T 86979833ec734d95bacb54c987af1e0a P 2e6115fe9cc74b2dad3b4948a436e22a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:17:38.857625 13447 raft_consensus.cc:482] T 86979833ec734d95bacb54c987af1e0a P 2e6115fe9cc74b2dad3b4948a436e22a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:17:38.857761 13447 raft_consensus.cc:3037] T 86979833ec734d95bacb54c987af1e0a P 2e6115fe9cc74b2dad3b4948a436e22a [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:17:38.860049 13447 raft_consensus.cc:504] T 86979833ec734d95bacb54c987af1e0a P 2e6115fe9cc74b2dad3b4948a436e22a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dd823ded93f47f9bc48fb5795e45b60" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 38817 } } peers { permanent_uuid: "2e6115fe9cc74b2dad3b4948a436e22a" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 44357 } } peers { permanent_uuid: "3f33588d4ac44320875c77d559c352a6" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 41901 } }
I20260807 18:17:38.861016 12979 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86979833ec734d95bacb54c987af1e0a" candidate_uuid: "2e6115fe9cc74b2dad3b4948a436e22a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5dd823ded93f47f9bc48fb5795e45b60"
I20260807 18:17:38.860706 13447 leader_election.cc:288] T 86979833ec734d95bacb54c987af1e0a P 2e6115fe9cc74b2dad3b4948a436e22a [CANDIDATE]: Term 1 election: Requested vote from peers 5dd823ded93f47f9bc48fb5795e45b60 (127.18.154.1:38817), 3f33588d4ac44320875c77d559c352a6 (127.18.154.3:41901)
I20260807 18:17:38.861168 13350 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86979833ec734d95bacb54c987af1e0a" candidate_uuid: "2e6115fe9cc74b2dad3b4948a436e22a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f33588d4ac44320875c77d559c352a6"
I20260807 18:17:38.861241 12979 raft_consensus.cc:3037] T 86979833ec734d95bacb54c987af1e0a P 5dd823ded93f47f9bc48fb5795e45b60 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:17:38.861383 13350 raft_consensus.cc:3037] T 86979833ec734d95bacb54c987af1e0a P 3f33588d4ac44320875c77d559c352a6 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:17:38.863561 12979 raft_consensus.cc:2454] T 86979833ec734d95bacb54c987af1e0a P 5dd823ded93f47f9bc48fb5795e45b60 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2e6115fe9cc74b2dad3b4948a436e22a in term 1.
I20260807 18:17:38.863678 13350 raft_consensus.cc:2454] T 86979833ec734d95bacb54c987af1e0a P 3f33588d4ac44320875c77d559c352a6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2e6115fe9cc74b2dad3b4948a436e22a in term 1.
I20260807 18:17:38.863914 13079 leader_election.cc:302] T 86979833ec734d95bacb54c987af1e0a P 2e6115fe9cc74b2dad3b4948a436e22a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2e6115fe9cc74b2dad3b4948a436e22a, 5dd823ded93f47f9bc48fb5795e45b60; no voters:
I20260807 18:17:38.864228 13447 raft_consensus.cc:2781] T 86979833ec734d95bacb54c987af1e0a P 2e6115fe9cc74b2dad3b4948a436e22a [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:17:38.864701 13447 raft_consensus.cc:686] T 86979833ec734d95bacb54c987af1e0a P 2e6115fe9cc74b2dad3b4948a436e22a [term 1 LEADER]: Becoming Leader. State: Replica: 2e6115fe9cc74b2dad3b4948a436e22a, State: Running, Role: LEADER
I20260807 18:17:38.865263 13447 consensus_queue.cc:237] T 86979833ec734d95bacb54c987af1e0a P 2e6115fe9cc74b2dad3b4948a436e22a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dd823ded93f47f9bc48fb5795e45b60" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 38817 } } peers { permanent_uuid: "2e6115fe9cc74b2dad3b4948a436e22a" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 44357 } } peers { permanent_uuid: "3f33588d4ac44320875c77d559c352a6" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 41901 } }
I20260807 18:17:38.869957 12783 catalog_manager.cc:5495] T 86979833ec734d95bacb54c987af1e0a P 2e6115fe9cc74b2dad3b4948a436e22a reported cstate change: term changed from 0 to 1, leader changed from <none> to 2e6115fe9cc74b2dad3b4948a436e22a (127.18.154.2). New cstate: current_term: 1 leader_uuid: "2e6115fe9cc74b2dad3b4948a436e22a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dd823ded93f47f9bc48fb5795e45b60" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 38817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2e6115fe9cc74b2dad3b4948a436e22a" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 44357 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3f33588d4ac44320875c77d559c352a6" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 41901 } health_report { overall_health: UNKNOWN } } }
W20260807 18:17:38.888892 13413 tablet.cc:2369] T 86979833ec734d95bacb54c987af1e0a P 3f33588d4ac44320875c77d559c352a6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:17:38.910514 13434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:38.905431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41794 (local address 127.18.154.2:44357)
0807 18:17:38.905638 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:38.905643 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:38.905661 (+ 18us) server_negotiation.cc:407] Connection header received
0807 18:17:38.905762 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:38.905765 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:38.905841 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:38.905966 (+ 125us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:17:38.906316 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:38.907050 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:38.908406 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:38.908796 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:38.908883 (+ 87us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:38.909333 (+ 450us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:17:38.909903 (+ 570us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:17:38.910036 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:38.910173 (+ 137us) server_negotiation.cc:299] Negotiation successful
0807 18:17:38.910235 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 18:17:38.918581 13350 raft_consensus.cc:1261] T 86979833ec734d95bacb54c987af1e0a P 3f33588d4ac44320875c77d559c352a6 [term 1 FOLLOWER]: Refusing update from remote peer 2e6115fe9cc74b2dad3b4948a436e22a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:17:38.918581 12979 raft_consensus.cc:1261] T 86979833ec734d95bacb54c987af1e0a P 5dd823ded93f47f9bc48fb5795e45b60 [term 1 FOLLOWER]: Refusing update from remote peer 2e6115fe9cc74b2dad3b4948a436e22a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:17:38.919623 13454 consensus_queue.cc:1035] T 86979833ec734d95bacb54c987af1e0a P 2e6115fe9cc74b2dad3b4948a436e22a [LEADER]: Connected to new peer: Peer: permanent_uuid: "5dd823ded93f47f9bc48fb5795e45b60" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 38817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:17:38.919975 13447 consensus_queue.cc:1035] T 86979833ec734d95bacb54c987af1e0a P 2e6115fe9cc74b2dad3b4948a436e22a [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f33588d4ac44320875c77d559c352a6" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 41901 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:17:38.935746 13462 mvcc.cc:205] Tried to move back new op lower bound from 7315974794920620032 to 7315974794716147712. Current Snapshot: MvccSnapshot[applied={T|T < 7315974794920620032}]
I20260807 18:17:38.944485 12782 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58200:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:17:38.944823 12782 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58200:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:17:38.948716 12782 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7e43081825ea4f97a226f34b4acfac54: Sending DeleteTablet for 3 replicas of tablet 86979833ec734d95bacb54c987af1e0a
I20260807 18:17:38.949942 12952 tablet_service.cc:1514] Processing DeleteTablet for tablet 86979833ec734d95bacb54c987af1e0a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:17:38 UTC) from {username='kudu', principal='kudu/127.18.154.62@KRBTEST.COM'} at 127.0.0.1:53176
I20260807 18:17:38.950049 13139 tablet_service.cc:1514] Processing DeleteTablet for tablet 86979833ec734d95bacb54c987af1e0a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:17:38 UTC) from {username='kudu', principal='kudu/127.18.154.62@KRBTEST.COM'} at 127.0.0.1:41792
I20260807 18:17:38.950218 13327 tablet_service.cc:1514] Processing DeleteTablet for tablet 86979833ec734d95bacb54c987af1e0a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:17:38 UTC) from {username='kudu', principal='kudu/127.18.154.62@KRBTEST.COM'} at 127.0.0.1:47690
I20260807 18:17:38.952236 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 12869
I20260807 18:17:38.960507 13474 tablet_replica.cc:330] T 86979833ec734d95bacb54c987af1e0a P 2e6115fe9cc74b2dad3b4948a436e22a: stopping tablet replica
I20260807 18:17:38.961108 13474 raft_consensus.cc:2229] T 86979833ec734d95bacb54c987af1e0a P 2e6115fe9cc74b2dad3b4948a436e22a [term 1 LEADER]: Raft consensus shutting down.
I20260807 18:17:38.961982 13474 raft_consensus.cc:2258] T 86979833ec734d95bacb54c987af1e0a P 2e6115fe9cc74b2dad3b4948a436e22a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:17:38.962841 13473 tablet_replica.cc:330] T 86979833ec734d95bacb54c987af1e0a P 5dd823ded93f47f9bc48fb5795e45b60: stopping tablet replica
I20260807 18:17:38.963315 13473 raft_consensus.cc:2229] T 86979833ec734d95bacb54c987af1e0a P 5dd823ded93f47f9bc48fb5795e45b60 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:17:38.963902 13473 raft_consensus.cc:2258] T 86979833ec734d95bacb54c987af1e0a P 5dd823ded93f47f9bc48fb5795e45b60 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:17:38.966408 13473 ts_tablet_manager.cc:1905] T 86979833ec734d95bacb54c987af1e0a P 5dd823ded93f47f9bc48fb5795e45b60: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:17:38.968127 13474 ts_tablet_manager.cc:1905] T 86979833ec734d95bacb54c987af1e0a P 2e6115fe9cc74b2dad3b4948a436e22a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:17:38.973680 13473 ts_tablet_manager.cc:1918] T 86979833ec734d95bacb54c987af1e0a P 5dd823ded93f47f9bc48fb5795e45b60: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:17:38.973930 13473 log.cc:1196] T 86979833ec734d95bacb54c987af1e0a P 5dd823ded93f47f9bc48fb5795e45b60: Deleting WAL directory at /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126328808105-19048-0/minicluster-data/ts-0/wal/wals/86979833ec734d95bacb54c987af1e0a
I20260807 18:17:38.974609 13473 ts_tablet_manager.cc:1939] T 86979833ec734d95bacb54c987af1e0a P 5dd823ded93f47f9bc48fb5795e45b60: Deleting consensus metadata
I20260807 18:17:38.976763 12770 catalog_manager.cc:4841] TS 5dd823ded93f47f9bc48fb5795e45b60 (127.18.154.1:38817): tablet 86979833ec734d95bacb54c987af1e0a (table test-table [id=edc6a82a4f1040a1a2eb9e0043a5c24c]) successfully deleted
I20260807 18:17:38.982370 13474 ts_tablet_manager.cc:1918] T 86979833ec734d95bacb54c987af1e0a P 2e6115fe9cc74b2dad3b4948a436e22a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:17:38.983109 13475 tablet_replica.cc:330] T 86979833ec734d95bacb54c987af1e0a P 3f33588d4ac44320875c77d559c352a6: stopping tablet replica
I20260807 18:17:38.983629 13474 log.cc:1196] T 86979833ec734d95bacb54c987af1e0a P 2e6115fe9cc74b2dad3b4948a436e22a: Deleting WAL directory at /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126328808105-19048-0/minicluster-data/ts-1/wal/wals/86979833ec734d95bacb54c987af1e0a
I20260807 18:17:38.983636 13475 raft_consensus.cc:2229] T 86979833ec734d95bacb54c987af1e0a P 3f33588d4ac44320875c77d559c352a6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:17:38.984306 13475 raft_consensus.cc:2258] T 86979833ec734d95bacb54c987af1e0a P 3f33588d4ac44320875c77d559c352a6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:17:38.984368 13474 ts_tablet_manager.cc:1939] T 86979833ec734d95bacb54c987af1e0a P 2e6115fe9cc74b2dad3b4948a436e22a: Deleting consensus metadata
I20260807 18:17:38.986569 12768 catalog_manager.cc:4841] TS 2e6115fe9cc74b2dad3b4948a436e22a (127.18.154.2:44357): tablet 86979833ec734d95bacb54c987af1e0a (table test-table [id=edc6a82a4f1040a1a2eb9e0043a5c24c]) successfully deleted
I20260807 18:17:38.986835 13475 ts_tablet_manager.cc:1905] T 86979833ec734d95bacb54c987af1e0a P 3f33588d4ac44320875c77d559c352a6: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 07 18:17:38 dist-test-slave-gkw7 krb5kdc[12715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126656, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.154.1@KRBTEST.COM
I20260807 18:17:38.994326 13475 ts_tablet_manager.cc:1918] T 86979833ec734d95bacb54c987af1e0a P 3f33588d4ac44320875c77d559c352a6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:17:38.994500 13475 log.cc:1196] T 86979833ec734d95bacb54c987af1e0a P 3f33588d4ac44320875c77d559c352a6: Deleting WAL directory at /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786126328808105-19048-0/minicluster-data/ts-2/wal/wals/86979833ec734d95bacb54c987af1e0a
I20260807 18:17:38.995114 13475 ts_tablet_manager.cc:1939] T 86979833ec734d95bacb54c987af1e0a P 3f33588d4ac44320875c77d559c352a6: Deleting consensus metadata
I20260807 18:17:38.995406 13433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:38.954116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53190 (local address 127.18.154.1:38817)
0807 18:17:38.954363 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:38.954372 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:38.975112 (+ 20740us) server_negotiation.cc:407] Connection header received
0807 18:17:38.975729 (+ 617us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:38.975738 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:38.975861 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:38.975956 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:17:38.978464 (+ 2508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:38.979986 (+ 1522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:38.982723 (+ 2737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:38.983225 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:38.983332 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:38.988930 (+ 5598us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:17:38.988943 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:17:38.988979 (+ 36us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:17:38.989032 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:17:38.992213 (+ 3181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:17:38.992917 (+ 704us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:17:38.992925 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:17:38.992927 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:17:38.993057 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:17:38.993468 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:17:38.993518 (+ 50us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:17:38.993521 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:17:38.993973 (+ 452us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:17:38.994287 (+ 314us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:38.994740 (+ 453us) server_negotiation.cc:299] Negotiation successful
0807 18:17:38.995035 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260807 18:17:38.996949 13036 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:17:38.997233 12768 catalog_manager.cc:4841] TS 3f33588d4ac44320875c77d559c352a6 (127.18.154.3:41901): tablet 86979833ec734d95bacb54c987af1e0a (table test-table [id=edc6a82a4f1040a1a2eb9e0043a5c24c]) successfully deleted
I20260807 18:17:39.111217 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 12869
I20260807 18:17:39.136967 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 13055
Aug 07 18:17:39 dist-test-slave-gkw7 krb5kdc[12715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126656, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.154.2@KRBTEST.COM
I20260807 18:17:39.153730 13434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:39.137954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41802 (local address 127.18.154.2:44357)
0807 18:17:39.138170 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:39.138176 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:39.138197 (+ 21us) server_negotiation.cc:407] Connection header received
0807 18:17:39.138331 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:39.138336 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:39.138401 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:39.138526 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:17:39.139831 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:39.140727 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:39.142299 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:39.142748 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:39.142886 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:39.147427 (+ 4541us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:17:39.147443 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:17:39.147460 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:17:39.147497 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:17:39.150728 (+ 3231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:17:39.151440 (+ 712us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:17:39.151448 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:17:39.151451 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:17:39.151600 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:17:39.151995 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:17:39.152000 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:17:39.152007 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:17:39.152417 (+ 410us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:17:39.152797 (+ 380us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:39.153034 (+ 237us) server_negotiation.cc:299] Negotiation successful
0807 18:17:39.153280 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260807 18:17:39.154533 13222 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:17:39.272332 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 13055
I20260807 18:17:39.300303 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 13236
Aug 07 18:17:39 dist-test-slave-gkw7 krb5kdc[12715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126656, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.154.3@KRBTEST.COM
I20260807 18:17:39.316247 13436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:39.301049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47700 (local address 127.18.154.3:41901)
0807 18:17:39.301236 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:39.301245 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:39.301322 (+ 77us) server_negotiation.cc:407] Connection header received
0807 18:17:39.301550 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:39.301556 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:39.301650 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:39.301770 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:17:39.303172 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:39.303910 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:39.305230 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:39.305739 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:39.305875 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:39.310840 (+ 4965us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:17:39.310856 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:17:39.310874 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:17:39.310910 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:17:39.313845 (+ 2935us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:17:39.314467 (+ 622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:17:39.314474 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:17:39.314476 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:17:39.314613 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:17:39.314942 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:17:39.314946 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:17:39.314948 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:17:39.315311 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:17:39.315519 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:39.315782 (+ 263us) server_negotiation.cc:299] Negotiation successful
0807 18:17:39.315949 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260807 18:17:39.316910 13403 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:17:39.408201 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 13236
I20260807 18:17:39.430051 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 12735
I20260807 18:17:39.431205 12824 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:17:39.541600 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 12735
2026-08-07T18:17:39Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (4786 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-07T18:17:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:17:39Z Disabled control of system clock
I20260807 18:17:39.639235 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126328808105-19048-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126328808105-19048-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126328808105-19048-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.154.62:37107
--webserver_interface=127.18.154.62
--webserver_port=0
--builtin_ntp_servers=127.18.154.20:38479
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.154.62:37107
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 18:17:39.993286 13490 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:17:39.993595 13490 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:17:39.993681 13490 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:17:40.001019 13490 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:17:40.001140 13490 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:17:40.001181 13490 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:17:40.001240 13490 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:17:40.001286 13490 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:17:40.010636 13490 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.154.20:38479
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126328808105-19048-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126328808105-19048-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.154.62:37107
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.154.62:37107
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.154.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126328808105-19048-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:17:40.011976 13490 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:17:40.013753 13490 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:17:40.026207 13497 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:17:40.026242 13498 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:17:40.026149 13501 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:17:40.027151 13490 server_base.cc:1034] running on GCE node
I20260807 18:17:40.027894 13490 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:17:40.029265 13490 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:17:40.030462 13490 hybrid_clock.cc:648] HybridClock initialized: now 1786126660030383 us; error 68 us; skew 500 ppm
I20260807 18:17:40.030836 13490 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 18:17:40.033143 13490 webserver.cc:460] Webserver started at http://127.18.154.62:42087/ using document root <none> and password file <none>
I20260807 18:17:40.033931 13490 fs_manager.cc:359] Metadata directory not provided
I20260807 18:17:40.034044 13490 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:17:40.034400 13490 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:17:40.037395 13490 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126328808105-19048-0/minicluster-data/master-0/data/instance:
uuid: "d02d0288512d4bcb8422f220de3c1c2a"
format_stamp: "Formatted at 2026-08-07 18:17:40 on dist-test-slave-gkw7"
server_key: "69839a0e96a78a3f747c7bd3be5bd56f"
server_key_iv: "f101809506c1fa9a5963607a07fe5984"
server_key_version: "encryptionkey@0"
I20260807 18:17:40.038281 13490 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126328808105-19048-0/minicluster-data/master-0/wal/instance:
uuid: "d02d0288512d4bcb8422f220de3c1c2a"
format_stamp: "Formatted at 2026-08-07 18:17:40 on dist-test-slave-gkw7"
server_key: "69839a0e96a78a3f747c7bd3be5bd56f"
server_key_iv: "f101809506c1fa9a5963607a07fe5984"
server_key_version: "encryptionkey@0"
I20260807 18:17:40.043452 13490 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.002s
I20260807 18:17:40.046975 13509 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:17:40.048382 13490 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 18:17:40.048593 13490 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126328808105-19048-0/minicluster-data/master-0/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126328808105-19048-0/minicluster-data/master-0/wal
uuid: "d02d0288512d4bcb8422f220de3c1c2a"
format_stamp: "Formatted at 2026-08-07 18:17:40 on dist-test-slave-gkw7"
server_key: "69839a0e96a78a3f747c7bd3be5bd56f"
server_key_iv: "f101809506c1fa9a5963607a07fe5984"
server_key_version: "encryptionkey@0"
I20260807 18:17:40.048838 13490 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126328808105-19048-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126328808105-19048-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126328808105-19048-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:17:40.065451 13490 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:17:40.066383 13490 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:17:40.066685 13490 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:17:40.085913 13490 rpc_server.cc:307] RPC server started. Bound to: 127.18.154.62:37107
I20260807 18:17:40.085909 13578 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.154.62:37107 every 8 connection(s)
I20260807 18:17:40.087358 13490 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
I20260807 18:17:40.091290 13579 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:17:40.092031 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 13490
I20260807 18:17:40.092208 19048 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126328808105-19048-0/minicluster-data/master-0/wal/instance
I20260807 18:17:40.092692 19048 external_mini_cluster.cc:1428] Setting key 43a9b024bc8da0155e5651f99471ff45
I20260807 18:17:40.106592 13579 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d02d0288512d4bcb8422f220de3c1c2a: Bootstrap starting.
I20260807 18:17:40.107740 13587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:40.094740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36486 (local address 127.18.154.62:37107)
0807 18:17:40.095872 (+ 1132us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:40.095892 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:40.095947 (+ 55us) server_negotiation.cc:407] Connection header received
0807 18:17:40.098406 (+ 2459us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:40.098441 (+ 35us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:40.099100 (+ 659us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:40.099468 (+ 368us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:17:40.100141 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:40.101396 (+ 1255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:40.103360 (+ 1964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:40.104057 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:40.104191 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:40.105418 (+ 1227us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:17:40.105425 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:17:40.105439 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:17:40.106033 (+ 594us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:17:40.106113 (+ 80us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:17:40.106134 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:17:40.106237 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:40.106463 (+ 226us) server_negotiation.cc:299] Negotiation successful
0807 18:17:40.106799 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":587,"thread_start_us":364,"threads_started":1}
I20260807 18:17:40.110316 13579 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d02d0288512d4bcb8422f220de3c1c2a: Neither blocks nor log segments found. Creating new log.
I20260807 18:17:40.111346 13579 log.cc:824] T 00000000000000000000000000000000 P d02d0288512d4bcb8422f220de3c1c2a: Log is configured to *not* fsync() on all Append() calls
I20260807 18:17:40.113968 13579 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d02d0288512d4bcb8422f220de3c1c2a: No bootstrap required, opened a new log
I20260807 18:17:40.121340 13579 raft_consensus.cc:348] T 00000000000000000000000000000000 P d02d0288512d4bcb8422f220de3c1c2a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d02d0288512d4bcb8422f220de3c1c2a" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 37107 } }
I20260807 18:17:40.121739 13579 raft_consensus.cc:374] T 00000000000000000000000000000000 P d02d0288512d4bcb8422f220de3c1c2a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:17:40.121837 13579 raft_consensus.cc:729] T 00000000000000000000000000000000 P d02d0288512d4bcb8422f220de3c1c2a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d02d0288512d4bcb8422f220de3c1c2a, State: Initialized, Role: FOLLOWER
I20260807 18:17:40.122332 13579 consensus_queue.cc:260] T 00000000000000000000000000000000 P d02d0288512d4bcb8422f220de3c1c2a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d02d0288512d4bcb8422f220de3c1c2a" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 37107 } }
I20260807 18:17:40.122535 13579 raft_consensus.cc:388] T 00000000000000000000000000000000 P d02d0288512d4bcb8422f220de3c1c2a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:17:40.122632 13579 raft_consensus.cc:482] T 00000000000000000000000000000000 P d02d0288512d4bcb8422f220de3c1c2a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:17:40.122761 13579 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d02d0288512d4bcb8422f220de3c1c2a [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:17:40.124866 13579 raft_consensus.cc:504] T 00000000000000000000000000000000 P d02d0288512d4bcb8422f220de3c1c2a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d02d0288512d4bcb8422f220de3c1c2a" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 37107 } }
I20260807 18:17:40.125346 13579 leader_election.cc:302] T 00000000000000000000000000000000 P d02d0288512d4bcb8422f220de3c1c2a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d02d0288512d4bcb8422f220de3c1c2a; no voters:
I20260807 18:17:40.126084 13579 leader_election.cc:288] T 00000000000000000000000000000000 P d02d0288512d4bcb8422f220de3c1c2a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:17:40.126215 13590 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d02d0288512d4bcb8422f220de3c1c2a [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:17:40.127018 13590 raft_consensus.cc:686] T 00000000000000000000000000000000 P d02d0288512d4bcb8422f220de3c1c2a [term 1 LEADER]: Becoming Leader. State: Replica: d02d0288512d4bcb8422f220de3c1c2a, State: Running, Role: LEADER
I20260807 18:17:40.127599 13590 consensus_queue.cc:237] T 00000000000000000000000000000000 P d02d0288512d4bcb8422f220de3c1c2a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d02d0288512d4bcb8422f220de3c1c2a" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 37107 } }
I20260807 18:17:40.128300 13579 sys_catalog.cc:564] T 00000000000000000000000000000000 P d02d0288512d4bcb8422f220de3c1c2a [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:17:40.130681 13592 sys_catalog.cc:455] T 00000000000000000000000000000000 P d02d0288512d4bcb8422f220de3c1c2a [sys.catalog]: SysCatalogTable state changed. Reason: New leader d02d0288512d4bcb8422f220de3c1c2a. Latest consensus state: current_term: 1 leader_uuid: "d02d0288512d4bcb8422f220de3c1c2a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d02d0288512d4bcb8422f220de3c1c2a" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 37107 } } }
I20260807 18:17:40.131199 13592 sys_catalog.cc:458] T 00000000000000000000000000000000 P d02d0288512d4bcb8422f220de3c1c2a [sys.catalog]: This master's current role is: LEADER
I20260807 18:17:40.130788 13591 sys_catalog.cc:455] T 00000000000000000000000000000000 P d02d0288512d4bcb8422f220de3c1c2a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d02d0288512d4bcb8422f220de3c1c2a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d02d0288512d4bcb8422f220de3c1c2a" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 37107 } } }
I20260807 18:17:40.131624 13591 sys_catalog.cc:458] T 00000000000000000000000000000000 P d02d0288512d4bcb8422f220de3c1c2a [sys.catalog]: This master's current role is: LEADER
I20260807 18:17:40.135041 13600 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:17:40.139909 13600 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:17:40.148976 13600 catalog_manager.cc:1347] Generated new cluster ID: f1c629c4564e46b796604274a2513eb8
I20260807 18:17:40.149101 13600 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:17:40.168660 13600 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:17:40.170516 13600 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:17:40.176962 13600 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d02d0288512d4bcb8422f220de3c1c2a: Generated new TSK 0
I20260807 18:17:40.177866 13600 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:17:40.183756 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126328808105-19048-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126328808105-19048-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126328808105-19048-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126328808105-19048-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.154.1:0
--local_ip_for_outbound_sockets=127.18.154.1
--webserver_interface=127.18.154.1
--webserver_port=0
--tserver_master_addrs=127.18.154.62:37107
--builtin_ntp_servers=127.18.154.20:38479
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 18:17:40.543169 13613 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:17:40.543495 13613 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:17:40.543659 13613 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:17:40.550972 13613 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:17:40.551081 13613 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:17:40.551292 13613 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.154.1
I20260807 18:17:40.560776 13613 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.154.20:38479
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126328808105-19048-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126328808105-19048-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.154.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126328808105-19048-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.154.1
--webserver_port=0
--tserver_master_addrs=127.18.154.62:37107
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.154.1
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126328808105-19048-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:17:40.562175 13613 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:17:40.564023 13613 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:17:40.577783 13619 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:17:40.577816 13620 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:17:40.577741 13622 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:17:40.578613 13613 server_base.cc:1034] running on GCE node
I20260807 18:17:40.579241 13613 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:17:40.580237 13613 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:17:40.581475 13613 hybrid_clock.cc:648] HybridClock initialized: now 1786126660581425 us; error 47 us; skew 500 ppm
I20260807 18:17:40.581871 13613 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 18:17:40.584321 13613 webserver.cc:460] Webserver started at http://127.18.154.1:41729/ using document root <none> and password file <none>
I20260807 18:17:40.585054 13613 fs_manager.cc:359] Metadata directory not provided
I20260807 18:17:40.585186 13613 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:17:40.585518 13613 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:17:40.588449 13613 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126328808105-19048-0/minicluster-data/ts-0/data/instance:
uuid: "399de1a55cf8493b843174e3fc0f0d1f"
format_stamp: "Formatted at 2026-08-07 18:17:40 on dist-test-slave-gkw7"
server_key: "4f73db40fb243af7af0af5109f3cc29d"
server_key_iv: "c6441ef3692f86efa0e5e5651b75193f"
server_key_version: "encryptionkey@0"
I20260807 18:17:40.589272 13613 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126328808105-19048-0/minicluster-data/ts-0/wal/instance:
uuid: "399de1a55cf8493b843174e3fc0f0d1f"
format_stamp: "Formatted at 2026-08-07 18:17:40 on dist-test-slave-gkw7"
server_key: "4f73db40fb243af7af0af5109f3cc29d"
server_key_iv: "c6441ef3692f86efa0e5e5651b75193f"
server_key_version: "encryptionkey@0"
I20260807 18:17:40.594513 13613 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.002s
I20260807 18:17:40.598073 13633 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:17:40.599452 13613 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260807 18:17:40.599628 13613 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126328808105-19048-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126328808105-19048-0/minicluster-data/ts-0/wal
uuid: "399de1a55cf8493b843174e3fc0f0d1f"
format_stamp: "Formatted at 2026-08-07 18:17:40 on dist-test-slave-gkw7"
server_key: "4f73db40fb243af7af0af5109f3cc29d"
server_key_iv: "c6441ef3692f86efa0e5e5651b75193f"
server_key_version: "encryptionkey@0"
I20260807 18:17:40.599852 13613 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126328808105-19048-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126328808105-19048-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126328808105-19048-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:17:40.621610 13613 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:17:40.622543 13613 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:17:40.622884 13613 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:17:40.623926 13613 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:17:40.625463 13613 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:17:40.625612 13613 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260807 18:17:40.625746 13613 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:17:40.625835 13613 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:17:40.661665 13613 rpc_server.cc:307] RPC server started. Bound to: 127.18.154.1:41265
I20260807 18:17:40.661741 13811 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.154.1:41265 every 8 connection(s)
I20260807 18:17:40.663079 13613 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126328808105-19048-0/minicluster-data/ts-0/data/info.pb
I20260807 18:17:40.666661 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 13613
I20260807 18:17:40.666846 19048 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126328808105-19048-0/minicluster-data/ts-0/wal/instance
I20260807 18:17:40.667588 19048 external_mini_cluster.cc:1428] Setting key 6559f16ad10e10dd8520df3ab516e8b7
I20260807 18:17:40.672225 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126328808105-19048-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126328808105-19048-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126328808105-19048-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126328808105-19048-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.154.2:0
--local_ip_for_outbound_sockets=127.18.154.2
--webserver_interface=127.18.154.2
--webserver_port=0
--tserver_master_addrs=127.18.154.62:37107
--builtin_ntp_servers=127.18.154.20:38479
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 18:17:40.687650 13816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:40.665692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.1:59187 (local address 127.18.154.62:37107)
0807 18:17:40.666481 (+ 789us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:40.666489 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:40.674748 (+ 8259us) server_negotiation.cc:407] Connection header received
0807 18:17:40.677905 (+ 3157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:40.677913 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:40.678011 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:40.678140 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:17:40.679799 (+ 1659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:40.681011 (+ 1212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:40.683821 (+ 2810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:40.684624 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:40.684771 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:40.686652 (+ 1881us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:17:40.686657 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:17:40.686661 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:17:40.686682 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:17:40.686726 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:17:40.686738 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:17:40.686874 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:40.687139 (+ 265us) server_negotiation.cc:299] Negotiation successful
0807 18:17:40.687248 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":475,"thread_start_us":297,"threads_started":1}
I20260807 18:17:40.689042 13817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:40.666576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:37107 (local address 127.18.154.1:59187)
0807 18:17:40.674570 (+ 7994us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:40.674626 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0807 18:17:40.677551 (+ 2925us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:17:40.678440 (+ 889us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:17:40.678455 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:17:40.678624 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:17:40.679593 (+ 969us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:40.679621 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:40.681207 (+ 1586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:40.681214 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:40.683649 (+ 2435us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:40.683666 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:40.684889 (+ 1223us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:40.684897 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:40.685275 (+ 378us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:40.686214 (+ 939us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 18:17:40.686219 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:17:40.686285 (+ 66us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 18:17:40.686414 (+ 129us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:17:40.686871 (+ 457us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:17:40.686885 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:17:40.686897 (+ 12us) client_negotiation.cc:769] Sending connection context
0807 18:17:40.687106 (+ 209us) client_negotiation.cc:240] Negotiation successful
0807 18:17:40.687563 (+ 457us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7269,"thread_start_us":7027,"threads_started":1}
I20260807 18:17:40.691027 13813 heartbeater.cc:347] Connected to a master server at 127.18.154.62:37107
I20260807 18:17:40.691388 13813 heartbeater.cc:464] Registering TS with master...
I20260807 18:17:40.692251 13813 heartbeater.cc:511] Master 127.18.154.62:37107 requested a full tablet report, sending...
I20260807 18:17:40.694785 13534 ts_manager.cc:194] Registered new tserver with Master: 399de1a55cf8493b843174e3fc0f0d1f (127.18.154.1:41265)
I20260807 18:17:40.696573 13534 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.154.1:59187
W20260807 18:17:41.048651 13821 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:17:41.048946 13821 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:17:41.049129 13821 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:17:41.056564 13821 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:17:41.056672 13821 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:17:41.056849 13821 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.154.2
I20260807 18:17:41.066267 13821 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.154.20:38479
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126328808105-19048-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126328808105-19048-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.154.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126328808105-19048-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.154.2
--webserver_port=0
--tserver_master_addrs=127.18.154.62:37107
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.154.2
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126328808105-19048-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:17:41.067643 13821 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:17:41.069514 13821 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:17:41.083340 13830 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:17:41.083308 13833 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:17:41.083371 13828 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:17:41.084035 13821 server_base.cc:1034] running on GCE node
I20260807 18:17:41.084717 13821 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:17:41.085819 13821 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:17:41.087064 13821 hybrid_clock.cc:648] HybridClock initialized: now 1786126661087006 us; error 53 us; skew 500 ppm
I20260807 18:17:41.087402 13821 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 18:17:41.089944 13821 webserver.cc:460] Webserver started at http://127.18.154.2:33001/ using document root <none> and password file <none>
I20260807 18:17:41.090696 13821 fs_manager.cc:359] Metadata directory not provided
I20260807 18:17:41.090828 13821 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:17:41.091166 13821 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:17:41.094167 13821 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126328808105-19048-0/minicluster-data/ts-1/data/instance:
uuid: "37946147a6bf4c67987556eaeaf55e2a"
format_stamp: "Formatted at 2026-08-07 18:17:41 on dist-test-slave-gkw7"
server_key: "5fd049a8a2b7c9c27a5d9010db7a7706"
server_key_iv: "e1468965caeb74d7a673ea18518ffaca"
server_key_version: "encryptionkey@0"
I20260807 18:17:41.094974 13821 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126328808105-19048-0/minicluster-data/ts-1/wal/instance:
uuid: "37946147a6bf4c67987556eaeaf55e2a"
format_stamp: "Formatted at 2026-08-07 18:17:41 on dist-test-slave-gkw7"
server_key: "5fd049a8a2b7c9c27a5d9010db7a7706"
server_key_iv: "e1468965caeb74d7a673ea18518ffaca"
server_key_version: "encryptionkey@0"
I20260807 18:17:41.100596 13821 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 18:17:41.104472 13840 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:17:41.105898 13821 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260807 18:17:41.106078 13821 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126328808105-19048-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126328808105-19048-0/minicluster-data/ts-1/wal
uuid: "37946147a6bf4c67987556eaeaf55e2a"
format_stamp: "Formatted at 2026-08-07 18:17:41 on dist-test-slave-gkw7"
server_key: "5fd049a8a2b7c9c27a5d9010db7a7706"
server_key_iv: "e1468965caeb74d7a673ea18518ffaca"
server_key_version: "encryptionkey@0"
I20260807 18:17:41.106328 13821 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126328808105-19048-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126328808105-19048-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126328808105-19048-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:17:41.127861 13821 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:17:41.128780 13821 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:17:41.129109 13821 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:17:41.130246 13821 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:17:41.131891 13821 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:17:41.132023 13821 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:17:41.132169 13821 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:17:41.132278 13821 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:17:41.166947 13821 rpc_server.cc:307] RPC server started. Bound to: 127.18.154.2:33339
I20260807 18:17:41.167047 13994 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.154.2:33339 every 8 connection(s)
I20260807 18:17:41.168411 13821 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126328808105-19048-0/minicluster-data/ts-1/data/info.pb
I20260807 18:17:41.169945 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 13821
I20260807 18:17:41.170101 19048 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126328808105-19048-0/minicluster-data/ts-1/wal/instance
I20260807 18:17:41.170549 19048 external_mini_cluster.cc:1428] Setting key 75fa6382889de3e85077ba3af1505d2c
I20260807 18:17:41.174068 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126328808105-19048-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126328808105-19048-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126328808105-19048-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126328808105-19048-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.154.3:0
--local_ip_for_outbound_sockets=127.18.154.3
--webserver_interface=127.18.154.3
--webserver_port=0
--tserver_master_addrs=127.18.154.62:37107
--builtin_ntp_servers=127.18.154.20:38479
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 18:17:41.183422 13816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:41.172737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.2:34513 (local address 127.18.154.62:37107)
0807 18:17:41.173009 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:41.173014 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:41.174795 (+ 1781us) server_negotiation.cc:407] Connection header received
0807 18:17:41.176627 (+ 1832us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:41.176633 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:41.176734 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:41.176847 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:17:41.178167 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:41.178910 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:41.180505 (+ 1595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:41.180910 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:41.181049 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:41.182539 (+ 1490us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:17:41.182544 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:17:41.182548 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:17:41.182568 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:17:41.182601 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:17:41.182613 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:17:41.182710 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:41.182963 (+ 253us) server_negotiation.cc:299] Negotiation successful
0807 18:17:41.183073 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135}
I20260807 18:17:41.184554 13997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:41.173469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:37107 (local address 127.18.154.2:34513)
0807 18:17:41.174625 (+ 1156us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:41.174664 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 18:17:41.176347 (+ 1683us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:17:41.177117 (+ 770us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:17:41.177131 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:17:41.177256 (+ 125us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:17:41.177995 (+ 739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:41.178013 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:41.179076 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:41.179082 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:41.180355 (+ 1273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:41.180367 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:41.181143 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:41.181151 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:41.181395 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:41.182195 (+ 800us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 18:17:41.182197 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:17:41.182242 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 18:17:41.182326 (+ 84us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:17:41.182727 (+ 401us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:17:41.182742 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:17:41.182754 (+ 12us) client_negotiation.cc:769] Sending connection context
0807 18:17:41.182935 (+ 181us) client_negotiation.cc:240] Negotiation successful
0807 18:17:41.183363 (+ 428us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":581,"thread_start_us":342,"threads_started":1}
I20260807 18:17:41.186745 13995 heartbeater.cc:347] Connected to a master server at 127.18.154.62:37107
I20260807 18:17:41.187129 13995 heartbeater.cc:464] Registering TS with master...
I20260807 18:17:41.188148 13995 heartbeater.cc:511] Master 127.18.154.62:37107 requested a full tablet report, sending...
I20260807 18:17:41.189800 13534 ts_manager.cc:194] Registered new tserver with Master: 37946147a6bf4c67987556eaeaf55e2a (127.18.154.2:33339)
I20260807 18:17:41.190671 13534 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.154.2:34513
W20260807 18:17:41.525484 13999 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:17:41.525861 13999 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:17:41.526022 13999 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:17:41.533689 13999 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:17:41.533826 13999 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:17:41.534006 13999 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.154.3
I20260807 18:17:41.543241 13999 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.154.20:38479
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126328808105-19048-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126328808105-19048-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.154.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126328808105-19048-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.154.3
--webserver_port=0
--tserver_master_addrs=127.18.154.62:37107
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.154.3
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126328808105-19048-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:17:41.544633 13999 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:17:41.546532 13999 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:17:41.560086 14006 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:17:41.560130 14010 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:17:41.560148 14005 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:17:41.561414 13999 server_base.cc:1034] running on GCE node
I20260807 18:17:41.562120 13999 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:17:41.563093 13999 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:17:41.564330 13999 hybrid_clock.cc:648] HybridClock initialized: now 1786126661564332 us; error 104 us; skew 500 ppm
I20260807 18:17:41.564687 13999 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 18:17:41.567111 13999 webserver.cc:460] Webserver started at http://127.18.154.3:33091/ using document root <none> and password file <none>
I20260807 18:17:41.567842 13999 fs_manager.cc:359] Metadata directory not provided
I20260807 18:17:41.567978 13999 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:17:41.568389 13999 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:17:41.571501 13999 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126328808105-19048-0/minicluster-data/ts-2/data/instance:
uuid: "aad8342212384388bab87b438d380ec7"
format_stamp: "Formatted at 2026-08-07 18:17:41 on dist-test-slave-gkw7"
server_key: "0970f0c7fdd6f70dd098dfba701cbe5e"
server_key_iv: "ff6fa07132992344400f6ce6ddae0c6b"
server_key_version: "encryptionkey@0"
I20260807 18:17:41.572245 13999 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126328808105-19048-0/minicluster-data/ts-2/wal/instance:
uuid: "aad8342212384388bab87b438d380ec7"
format_stamp: "Formatted at 2026-08-07 18:17:41 on dist-test-slave-gkw7"
server_key: "0970f0c7fdd6f70dd098dfba701cbe5e"
server_key_iv: "ff6fa07132992344400f6ce6ddae0c6b"
server_key_version: "encryptionkey@0"
I20260807 18:17:41.577337 13999 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260807 18:17:41.581019 14016 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:17:41.582468 13999 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:17:41.582638 13999 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126328808105-19048-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126328808105-19048-0/minicluster-data/ts-2/wal
uuid: "aad8342212384388bab87b438d380ec7"
format_stamp: "Formatted at 2026-08-07 18:17:41 on dist-test-slave-gkw7"
server_key: "0970f0c7fdd6f70dd098dfba701cbe5e"
server_key_iv: "ff6fa07132992344400f6ce6ddae0c6b"
server_key_version: "encryptionkey@0"
I20260807 18:17:41.582865 13999 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126328808105-19048-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126328808105-19048-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126328808105-19048-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:17:41.600575 13999 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:17:41.601477 13999 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:17:41.601859 13999 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:17:41.602898 13999 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:17:41.604604 13999 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:17:41.604719 13999 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:17:41.604830 13999 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:17:41.604892 13999 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:17:41.639417 13999 rpc_server.cc:307] RPC server started. Bound to: 127.18.154.3:43361
I20260807 18:17:41.639464 14179 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.154.3:43361 every 8 connection(s)
I20260807 18:17:41.640856 13999 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126328808105-19048-0/minicluster-data/ts-2/data/info.pb
I20260807 18:17:41.651074 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 13999
I20260807 18:17:41.651298 19048 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126328808105-19048-0/minicluster-data/ts-2/wal/instance
I20260807 18:17:41.651923 19048 external_mini_cluster.cc:1428] Setting key 235adaedd7fcdd27fab2f5905a369474
I20260807 18:17:41.653748 13816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:41.643548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.3:46723 (local address 127.18.154.62:37107)
0807 18:17:41.643753 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:41.643758 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:41.645198 (+ 1440us) server_negotiation.cc:407] Connection header received
0807 18:17:41.647019 (+ 1821us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:41.647023 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:41.647095 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:41.647186 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:17:41.648249 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:41.648978 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:41.650614 (+ 1636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:41.650998 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:41.651092 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:41.652794 (+ 1702us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:17:41.652799 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:17:41.652802 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:17:41.652822 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:17:41.652859 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:17:41.652871 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:17:41.652976 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:41.653251 (+ 275us) server_negotiation.cc:299] Negotiation successful
0807 18:17:41.653357 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260807 18:17:41.654891 14182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:41.644112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:37107 (local address 127.18.154.3:46723)
0807 18:17:41.645030 (+ 918us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:41.645074 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 18:17:41.646755 (+ 1681us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:17:41.647390 (+ 635us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:17:41.647398 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:17:41.647489 (+ 91us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:17:41.648092 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:41.648109 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:41.649111 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:41.649114 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:41.650475 (+ 1361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:41.650488 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:41.651264 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:41.651273 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:41.651592 (+ 319us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:41.652403 (+ 811us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 18:17:41.652407 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:17:41.652465 (+ 58us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 18:17:41.652569 (+ 104us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:17:41.652978 (+ 409us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:17:41.652994 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:17:41.653004 (+ 10us) client_negotiation.cc:769] Sending connection context
0807 18:17:41.653210 (+ 206us) client_negotiation.cc:240] Negotiation successful
0807 18:17:41.653660 (+ 450us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":470,"thread_start_us":277,"threads_started":1}
I20260807 18:17:41.656176 14180 heartbeater.cc:347] Connected to a master server at 127.18.154.62:37107
I20260807 18:17:41.656519 14180 heartbeater.cc:464] Registering TS with master...
I20260807 18:17:41.657325 14180 heartbeater.cc:511] Master 127.18.154.62:37107 requested a full tablet report, sending...
I20260807 18:17:41.658850 13534 ts_manager.cc:194] Registered new tserver with Master: aad8342212384388bab87b438d380ec7 (127.18.154.3:43361)
I20260807 18:17:41.659683 13534 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.154.3:46723
I20260807 18:17:41.667589 19048 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 18:17:41.677609 13816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:41.671820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36492 (local address 127.18.154.62:37107)
0807 18:17:41.671998 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:41.672003 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:41.672527 (+ 524us) server_negotiation.cc:407] Connection header received
0807 18:17:41.672667 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:41.672671 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:41.672734 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:41.672816 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:17:41.673312 (+ 496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:41.674079 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:41.675374 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:41.675773 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:41.675863 (+ 90us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:41.676910 (+ 1047us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:17:41.676915 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:17:41.676918 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:17:41.676937 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:17:41.676968 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:17:41.676980 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:17:41.677095 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:41.677266 (+ 171us) server_negotiation.cc:299] Negotiation successful
0807 18:17:41.677315 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260807 18:17:41.681788 13534 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36492:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 18:17:41.689333 13534 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:17:41.699937 13813 heartbeater.cc:503] Master 127.18.154.62:37107 was elected leader, sending a full tablet report...
I20260807 18:17:41.713922 14196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:41.702431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.1:41265 (local address 127.0.0.1:49394)
0807 18:17:41.703267 (+ 836us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:41.703313 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 18:17:41.703508 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:17:41.705953 (+ 2445us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:17:41.705965 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:17:41.706038 (+ 73us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:17:41.706506 (+ 468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:41.706523 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:41.708545 (+ 2022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:41.708553 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:41.710938 (+ 2385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:41.710952 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:41.712903 (+ 1951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:41.712910 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:41.713111 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:41.713167 (+ 56us) client_negotiation.cc:769] Sending connection context
0807 18:17:41.713329 (+ 162us) client_negotiation.cc:240] Negotiation successful
0807 18:17:41.713514 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":529,"mutex_wait_us":3,"thread_start_us":353,"threads_started":1}
I20260807 18:17:41.714018 14197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:41.702705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49394 (local address 127.18.154.1:41265)
0807 18:17:41.705099 (+ 2394us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:41.705111 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:41.705149 (+ 38us) server_negotiation.cc:407] Connection header received
0807 18:17:41.705266 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:41.705278 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:41.705631 (+ 353us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:41.705882 (+ 251us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:17:41.706834 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:41.708415 (+ 1581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:41.711175 (+ 2760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:41.712680 (+ 1505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:41.712834 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:41.713118 (+ 284us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:41.713410 (+ 292us) server_negotiation.cc:299] Negotiation successful
0807 18:17:41.713621 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2157,"thread_start_us":1937,"threads_started":1}
I20260807 18:17:41.714810 14198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:41.703006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.2:33339 (local address 127.0.0.1:60524)
0807 18:17:41.704254 (+ 1248us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:41.704277 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 18:17:41.704416 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:17:41.707815 (+ 3399us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:17:41.707819 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:17:41.707843 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:17:41.708216 (+ 373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:41.708232 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:41.710108 (+ 1876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:41.710116 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:41.712280 (+ 2164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:41.712293 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:41.714032 (+ 1739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:41.714040 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:41.714245 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:41.714275 (+ 30us) client_negotiation.cc:769] Sending connection context
0807 18:17:41.714336 (+ 61us) client_negotiation.cc:240] Negotiation successful
0807 18:17:41.714465 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1108,"mutex_wait_us":88,"thread_start_us":969,"threads_started":1}
I20260807 18:17:41.716137 14201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:41.703781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.3:43361 (local address 127.0.0.1:51498)
0807 18:17:41.707489 (+ 3708us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:41.707506 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 18:17:41.707630 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:17:41.708587 (+ 957us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:17:41.708591 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:17:41.708623 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:17:41.709139 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:41.709153 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:41.711122 (+ 1969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:41.711127 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:41.713258 (+ 2131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:41.713271 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:41.715325 (+ 2054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:41.715331 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:41.715519 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:41.715555 (+ 36us) client_negotiation.cc:769] Sending connection context
0807 18:17:41.715648 (+ 93us) client_negotiation.cc:240] Negotiation successful
0807 18:17:41.715784 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3597,"mutex_wait_us":27,"thread_start_us":3459,"threads_started":1}
I20260807 18:17:41.716313 14200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:41.703599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60524 (local address 127.18.154.2:33339)
0807 18:17:41.707168 (+ 3569us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:41.707179 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:41.707210 (+ 31us) server_negotiation.cc:407] Connection header received
0807 18:17:41.707328 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:41.707337 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:41.707590 (+ 253us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:41.708431 (+ 841us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:17:41.708645 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:41.709994 (+ 1349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:41.712466 (+ 2472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:41.713806 (+ 1340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:41.715396 (+ 1590us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:41.715654 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:41.715732 (+ 78us) server_negotiation.cc:299] Negotiation successful
0807 18:17:41.715916 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3409,"thread_start_us":3310,"threads_started":1}
I20260807 18:17:41.716933 14203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:41.704487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51498 (local address 127.18.154.3:43361)
0807 18:17:41.706739 (+ 2252us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:41.706751 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:41.707760 (+ 1009us) server_negotiation.cc:407] Connection header received
0807 18:17:41.707858 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:41.707868 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:41.708094 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:41.708303 (+ 209us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:17:41.709354 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:41.710939 (+ 1585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:41.713510 (+ 2571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:41.714791 (+ 1281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:41.714925 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:41.715179 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:41.716396 (+ 1217us) server_negotiation.cc:299] Negotiation successful
0807 18:17:41.716561 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2011,"thread_start_us":1825,"threads_started":1}
I20260807 18:17:41.718894 13722 tablet_service.cc:1467] Processing CreateTablet for tablet 132449bf61604c74993af0547ba745cb (DEFAULT_TABLE table=test-table [id=89a13cfae26344578ab8c53c01e0b17c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:17:41.719955 13906 tablet_service.cc:1467] Processing CreateTablet for tablet 132449bf61604c74993af0547ba745cb (DEFAULT_TABLE table=test-table [id=89a13cfae26344578ab8c53c01e0b17c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:17:41.720605 13722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 132449bf61604c74993af0547ba745cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:17:41.720321 14081 tablet_service.cc:1467] Processing CreateTablet for tablet 132449bf61604c74993af0547ba745cb (DEFAULT_TABLE table=test-table [id=89a13cfae26344578ab8c53c01e0b17c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:17:41.721361 13906 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 132449bf61604c74993af0547ba745cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:17:41.721729 14081 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 132449bf61604c74993af0547ba745cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:17:41.734035 14205 tablet_bootstrap.cc:492] T 132449bf61604c74993af0547ba745cb P aad8342212384388bab87b438d380ec7: Bootstrap starting.
I20260807 18:17:41.737437 14205 tablet_bootstrap.cc:654] T 132449bf61604c74993af0547ba745cb P aad8342212384388bab87b438d380ec7: Neither blocks nor log segments found. Creating new log.
I20260807 18:17:41.739491 14205 log.cc:824] T 132449bf61604c74993af0547ba745cb P aad8342212384388bab87b438d380ec7: Log is configured to *not* fsync() on all Append() calls
I20260807 18:17:41.739682 14208 tablet_bootstrap.cc:492] T 132449bf61604c74993af0547ba745cb P 399de1a55cf8493b843174e3fc0f0d1f: Bootstrap starting.
I20260807 18:17:41.741086 14206 tablet_bootstrap.cc:492] T 132449bf61604c74993af0547ba745cb P 37946147a6bf4c67987556eaeaf55e2a: Bootstrap starting.
I20260807 18:17:41.743747 14208 tablet_bootstrap.cc:654] T 132449bf61604c74993af0547ba745cb P 399de1a55cf8493b843174e3fc0f0d1f: Neither blocks nor log segments found. Creating new log.
I20260807 18:17:41.744771 14206 tablet_bootstrap.cc:654] T 132449bf61604c74993af0547ba745cb P 37946147a6bf4c67987556eaeaf55e2a: Neither blocks nor log segments found. Creating new log.
I20260807 18:17:41.745121 14208 log.cc:824] T 132449bf61604c74993af0547ba745cb P 399de1a55cf8493b843174e3fc0f0d1f: Log is configured to *not* fsync() on all Append() calls
I20260807 18:17:41.746073 14206 log.cc:824] T 132449bf61604c74993af0547ba745cb P 37946147a6bf4c67987556eaeaf55e2a: Log is configured to *not* fsync() on all Append() calls
I20260807 18:17:41.746978 14205 tablet_bootstrap.cc:492] T 132449bf61604c74993af0547ba745cb P aad8342212384388bab87b438d380ec7: No bootstrap required, opened a new log
I20260807 18:17:41.747584 14205 ts_tablet_manager.cc:1397] T 132449bf61604c74993af0547ba745cb P aad8342212384388bab87b438d380ec7: Time spent bootstrapping tablet: real 0.014s user 0.008s sys 0.000s
I20260807 18:17:41.748782 14208 tablet_bootstrap.cc:492] T 132449bf61604c74993af0547ba745cb P 399de1a55cf8493b843174e3fc0f0d1f: No bootstrap required, opened a new log
I20260807 18:17:41.749122 14208 ts_tablet_manager.cc:1397] T 132449bf61604c74993af0547ba745cb P 399de1a55cf8493b843174e3fc0f0d1f: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.000s
I20260807 18:17:41.755743 14205 raft_consensus.cc:348] T 132449bf61604c74993af0547ba745cb P aad8342212384388bab87b438d380ec7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "399de1a55cf8493b843174e3fc0f0d1f" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 41265 } } peers { permanent_uuid: "aad8342212384388bab87b438d380ec7" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 43361 } } peers { permanent_uuid: "37946147a6bf4c67987556eaeaf55e2a" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 33339 } }
I20260807 18:17:41.756057 14206 tablet_bootstrap.cc:492] T 132449bf61604c74993af0547ba745cb P 37946147a6bf4c67987556eaeaf55e2a: No bootstrap required, opened a new log
I20260807 18:17:41.756134 14205 raft_consensus.cc:374] T 132449bf61604c74993af0547ba745cb P aad8342212384388bab87b438d380ec7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:17:41.756283 14205 raft_consensus.cc:729] T 132449bf61604c74993af0547ba745cb P aad8342212384388bab87b438d380ec7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aad8342212384388bab87b438d380ec7, State: Initialized, Role: FOLLOWER
I20260807 18:17:41.756376 14206 ts_tablet_manager.cc:1397] T 132449bf61604c74993af0547ba745cb P 37946147a6bf4c67987556eaeaf55e2a: Time spent bootstrapping tablet: real 0.016s user 0.014s sys 0.000s
I20260807 18:17:41.756939 14205 consensus_queue.cc:260] T 132449bf61604c74993af0547ba745cb P aad8342212384388bab87b438d380ec7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "399de1a55cf8493b843174e3fc0f0d1f" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 41265 } } peers { permanent_uuid: "aad8342212384388bab87b438d380ec7" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 43361 } } peers { permanent_uuid: "37946147a6bf4c67987556eaeaf55e2a" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 33339 } }
I20260807 18:17:41.758100 14180 heartbeater.cc:503] Master 127.18.154.62:37107 was elected leader, sending a full tablet report...
I20260807 18:17:41.758647 14205 ts_tablet_manager.cc:1428] T 132449bf61604c74993af0547ba745cb P aad8342212384388bab87b438d380ec7: Time spent starting tablet: real 0.011s user 0.008s sys 0.001s
I20260807 18:17:41.760548 14208 raft_consensus.cc:348] T 132449bf61604c74993af0547ba745cb P 399de1a55cf8493b843174e3fc0f0d1f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "399de1a55cf8493b843174e3fc0f0d1f" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 41265 } } peers { permanent_uuid: "aad8342212384388bab87b438d380ec7" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 43361 } } peers { permanent_uuid: "37946147a6bf4c67987556eaeaf55e2a" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 33339 } }
I20260807 18:17:41.761265 14208 raft_consensus.cc:374] T 132449bf61604c74993af0547ba745cb P 399de1a55cf8493b843174e3fc0f0d1f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:17:41.761435 14208 raft_consensus.cc:729] T 132449bf61604c74993af0547ba745cb P 399de1a55cf8493b843174e3fc0f0d1f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 399de1a55cf8493b843174e3fc0f0d1f, State: Initialized, Role: FOLLOWER
I20260807 18:17:41.762140 14208 consensus_queue.cc:260] T 132449bf61604c74993af0547ba745cb P 399de1a55cf8493b843174e3fc0f0d1f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "399de1a55cf8493b843174e3fc0f0d1f" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 41265 } } peers { permanent_uuid: "aad8342212384388bab87b438d380ec7" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 43361 } } peers { permanent_uuid: "37946147a6bf4c67987556eaeaf55e2a" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 33339 } }
I20260807 18:17:41.764056 14208 ts_tablet_manager.cc:1428] T 132449bf61604c74993af0547ba745cb P 399de1a55cf8493b843174e3fc0f0d1f: Time spent starting tablet: real 0.015s user 0.007s sys 0.007s
I20260807 18:17:41.767292 14206 raft_consensus.cc:348] T 132449bf61604c74993af0547ba745cb P 37946147a6bf4c67987556eaeaf55e2a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "399de1a55cf8493b843174e3fc0f0d1f" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 41265 } } peers { permanent_uuid: "aad8342212384388bab87b438d380ec7" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 43361 } } peers { permanent_uuid: "37946147a6bf4c67987556eaeaf55e2a" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 33339 } }
I20260807 18:17:41.767763 14206 raft_consensus.cc:374] T 132449bf61604c74993af0547ba745cb P 37946147a6bf4c67987556eaeaf55e2a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:17:41.767897 14206 raft_consensus.cc:729] T 132449bf61604c74993af0547ba745cb P 37946147a6bf4c67987556eaeaf55e2a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 37946147a6bf4c67987556eaeaf55e2a, State: Initialized, Role: FOLLOWER
I20260807 18:17:41.768465 14206 consensus_queue.cc:260] T 132449bf61604c74993af0547ba745cb P 37946147a6bf4c67987556eaeaf55e2a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "399de1a55cf8493b843174e3fc0f0d1f" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 41265 } } peers { permanent_uuid: "aad8342212384388bab87b438d380ec7" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 43361 } } peers { permanent_uuid: "37946147a6bf4c67987556eaeaf55e2a" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 33339 } }
I20260807 18:17:41.769626 13995 heartbeater.cc:503] Master 127.18.154.62:37107 was elected leader, sending a full tablet report...
I20260807 18:17:41.770153 14206 ts_tablet_manager.cc:1428] T 132449bf61604c74993af0547ba745cb P 37946147a6bf4c67987556eaeaf55e2a: Time spent starting tablet: real 0.014s user 0.012s sys 0.000s
W20260807 18:17:41.892689 14181 tablet.cc:2369] T 132449bf61604c74993af0547ba745cb P aad8342212384388bab87b438d380ec7: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 18:17:41.915781 13815 tablet.cc:2369] T 132449bf61604c74993af0547ba745cb P 399de1a55cf8493b843174e3fc0f0d1f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 18:17:41.920605 13996 tablet.cc:2369] T 132449bf61604c74993af0547ba745cb P 37946147a6bf4c67987556eaeaf55e2a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:17:42.096921 14212 raft_consensus.cc:482] T 132449bf61604c74993af0547ba745cb P aad8342212384388bab87b438d380ec7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:17:42.097301 14212 raft_consensus.cc:504] T 132449bf61604c74993af0547ba745cb P aad8342212384388bab87b438d380ec7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "399de1a55cf8493b843174e3fc0f0d1f" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 41265 } } peers { permanent_uuid: "aad8342212384388bab87b438d380ec7" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 43361 } } peers { permanent_uuid: "37946147a6bf4c67987556eaeaf55e2a" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 33339 } }
I20260807 18:17:42.099161 14212 leader_election.cc:288] T 132449bf61604c74993af0547ba745cb P aad8342212384388bab87b438d380ec7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 399de1a55cf8493b843174e3fc0f0d1f (127.18.154.1:41265), 37946147a6bf4c67987556eaeaf55e2a (127.18.154.2:33339)
I20260807 18:17:42.106657 14200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:42.099998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.3:51441 (local address 127.18.154.2:33339)
0807 18:17:42.100250 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:42.100257 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:42.100283 (+ 26us) server_negotiation.cc:407] Connection header received
0807 18:17:42.100372 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:42.100379 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:42.100477 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:42.100600 (+ 123us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:17:42.101144 (+ 544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:42.102163 (+ 1019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:42.104571 (+ 2408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:42.105820 (+ 1249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:42.105973 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:42.106052 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:42.106302 (+ 250us) server_negotiation.cc:299] Negotiation successful
0807 18:17:42.106439 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
I20260807 18:17:42.106679 14182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:42.099530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.2:33339 (local address 127.18.154.3:51441)
0807 18:17:42.099701 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:42.099748 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 18:17:42.099884 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:17:42.100701 (+ 817us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:17:42.100708 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:17:42.100753 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:17:42.100982 (+ 229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:42.100998 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:42.102307 (+ 1309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:42.102313 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:42.104409 (+ 2096us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:42.104424 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:42.105976 (+ 1552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:42.105980 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:42.106121 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:42.106146 (+ 25us) client_negotiation.cc:769] Sending connection context
0807 18:17:42.106216 (+ 70us) client_negotiation.cc:240] Negotiation successful
0807 18:17:42.106342 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40,"mutex_wait_us":27}
I20260807 18:17:42.107179 14197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:42.099993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.3:40969 (local address 127.18.154.1:41265)
0807 18:17:42.100215 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:42.100234 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:42.100837 (+ 603us) server_negotiation.cc:407] Connection header received
0807 18:17:42.101033 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:42.101040 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:42.101142 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:42.101258 (+ 116us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:17:42.101839 (+ 581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:42.102590 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:42.104928 (+ 2338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:42.106241 (+ 1313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:42.106414 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:42.106474 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:42.106794 (+ 320us) server_negotiation.cc:299] Negotiation successful
0807 18:17:42.106939 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260807 18:17:42.107527 14215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:42.099553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.1:41265 (local address 127.18.154.3:40969)
0807 18:17:42.100701 (+ 1148us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:42.100717 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 18:17:42.100863 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:17:42.101286 (+ 423us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:17:42.101291 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:17:42.101315 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:17:42.101652 (+ 337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:42.101663 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:42.102735 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:42.102742 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:42.104763 (+ 2021us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:42.104773 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:42.106416 (+ 1643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:42.106422 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:42.106598 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:42.106637 (+ 39us) client_negotiation.cc:769] Sending connection context
0807 18:17:42.106736 (+ 99us) client_negotiation.cc:240] Negotiation successful
0807 18:17:42.107159 (+ 423us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1025,"mutex_wait_us":476,"spinlock_wait_cycles":27008,"thread_start_us":385,"threads_started":1}
I20260807 18:17:42.107829 13930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "132449bf61604c74993af0547ba745cb" candidate_uuid: "aad8342212384388bab87b438d380ec7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "37946147a6bf4c67987556eaeaf55e2a" is_pre_election: true
I20260807 18:17:42.108469 13930 raft_consensus.cc:2454] T 132449bf61604c74993af0547ba745cb P 37946147a6bf4c67987556eaeaf55e2a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aad8342212384388bab87b438d380ec7 in term 0.
I20260807 18:17:42.108624 13752 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "132449bf61604c74993af0547ba745cb" candidate_uuid: "aad8342212384388bab87b438d380ec7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "399de1a55cf8493b843174e3fc0f0d1f" is_pre_election: true
I20260807 18:17:42.109163 13752 raft_consensus.cc:2454] T 132449bf61604c74993af0547ba745cb P 399de1a55cf8493b843174e3fc0f0d1f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aad8342212384388bab87b438d380ec7 in term 0.
I20260807 18:17:42.109186 14021 leader_election.cc:302] T 132449bf61604c74993af0547ba745cb P aad8342212384388bab87b438d380ec7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 37946147a6bf4c67987556eaeaf55e2a, aad8342212384388bab87b438d380ec7; no voters:
I20260807 18:17:42.109640 14212 raft_consensus.cc:2781] T 132449bf61604c74993af0547ba745cb P aad8342212384388bab87b438d380ec7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:17:42.109838 14212 raft_consensus.cc:482] T 132449bf61604c74993af0547ba745cb P aad8342212384388bab87b438d380ec7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:17:42.109968 14212 raft_consensus.cc:3037] T 132449bf61604c74993af0547ba745cb P aad8342212384388bab87b438d380ec7 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:17:42.112418 14212 raft_consensus.cc:504] T 132449bf61604c74993af0547ba745cb P aad8342212384388bab87b438d380ec7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "399de1a55cf8493b843174e3fc0f0d1f" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 41265 } } peers { permanent_uuid: "aad8342212384388bab87b438d380ec7" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 43361 } } peers { permanent_uuid: "37946147a6bf4c67987556eaeaf55e2a" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 33339 } }
I20260807 18:17:42.113018 14212 leader_election.cc:288] T 132449bf61604c74993af0547ba745cb P aad8342212384388bab87b438d380ec7 [CANDIDATE]: Term 1 election: Requested vote from peers 399de1a55cf8493b843174e3fc0f0d1f (127.18.154.1:41265), 37946147a6bf4c67987556eaeaf55e2a (127.18.154.2:33339)
I20260807 18:17:42.113477 13752 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "132449bf61604c74993af0547ba745cb" candidate_uuid: "aad8342212384388bab87b438d380ec7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "399de1a55cf8493b843174e3fc0f0d1f"
I20260807 18:17:42.113596 13930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "132449bf61604c74993af0547ba745cb" candidate_uuid: "aad8342212384388bab87b438d380ec7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "37946147a6bf4c67987556eaeaf55e2a"
I20260807 18:17:42.113752 13752 raft_consensus.cc:3037] T 132449bf61604c74993af0547ba745cb P 399de1a55cf8493b843174e3fc0f0d1f [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:17:42.113826 13930 raft_consensus.cc:3037] T 132449bf61604c74993af0547ba745cb P 37946147a6bf4c67987556eaeaf55e2a [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:17:42.116009 13752 raft_consensus.cc:2454] T 132449bf61604c74993af0547ba745cb P 399de1a55cf8493b843174e3fc0f0d1f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aad8342212384388bab87b438d380ec7 in term 1.
I20260807 18:17:42.116021 13930 raft_consensus.cc:2454] T 132449bf61604c74993af0547ba745cb P 37946147a6bf4c67987556eaeaf55e2a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aad8342212384388bab87b438d380ec7 in term 1.
I20260807 18:17:42.116454 14019 leader_election.cc:302] T 132449bf61604c74993af0547ba745cb P aad8342212384388bab87b438d380ec7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 399de1a55cf8493b843174e3fc0f0d1f, aad8342212384388bab87b438d380ec7; no voters:
I20260807 18:17:42.116766 14212 raft_consensus.cc:2781] T 132449bf61604c74993af0547ba745cb P aad8342212384388bab87b438d380ec7 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:17:42.117225 14212 raft_consensus.cc:686] T 132449bf61604c74993af0547ba745cb P aad8342212384388bab87b438d380ec7 [term 1 LEADER]: Becoming Leader. State: Replica: aad8342212384388bab87b438d380ec7, State: Running, Role: LEADER
I20260807 18:17:42.117838 14212 consensus_queue.cc:237] T 132449bf61604c74993af0547ba745cb P aad8342212384388bab87b438d380ec7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "399de1a55cf8493b843174e3fc0f0d1f" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 41265 } } peers { permanent_uuid: "aad8342212384388bab87b438d380ec7" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 43361 } } peers { permanent_uuid: "37946147a6bf4c67987556eaeaf55e2a" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 33339 } }
I20260807 18:17:42.122498 13532 catalog_manager.cc:5495] T 132449bf61604c74993af0547ba745cb P aad8342212384388bab87b438d380ec7 reported cstate change: term changed from 0 to 1, leader changed from <none> to aad8342212384388bab87b438d380ec7 (127.18.154.3). New cstate: current_term: 1 leader_uuid: "aad8342212384388bab87b438d380ec7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "399de1a55cf8493b843174e3fc0f0d1f" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 41265 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aad8342212384388bab87b438d380ec7" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 43361 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "37946147a6bf4c67987556eaeaf55e2a" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 33339 } health_report { overall_health: UNKNOWN } } }
I20260807 18:17:42.180318 14203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:42.174748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51510 (local address 127.18.154.3:43361)
0807 18:17:42.174967 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:42.174973 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:42.174996 (+ 23us) server_negotiation.cc:407] Connection header received
0807 18:17:42.175115 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:42.175119 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:42.175208 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:42.175368 (+ 160us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:17:42.175785 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:42.176545 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:42.178036 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:42.178447 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:42.178591 (+ 144us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:42.179221 (+ 630us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:17:42.179684 (+ 463us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:17:42.179886 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:42.180038 (+ 152us) server_negotiation.cc:299] Negotiation successful
0807 18:17:42.180084 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260807 18:17:42.188874 13752 raft_consensus.cc:1261] T 132449bf61604c74993af0547ba745cb P 399de1a55cf8493b843174e3fc0f0d1f [term 1 FOLLOWER]: Refusing update from remote peer aad8342212384388bab87b438d380ec7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:17:42.188874 13930 raft_consensus.cc:1261] T 132449bf61604c74993af0547ba745cb P 37946147a6bf4c67987556eaeaf55e2a [term 1 FOLLOWER]: Refusing update from remote peer aad8342212384388bab87b438d380ec7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:17:42.190037 14212 consensus_queue.cc:1035] T 132449bf61604c74993af0547ba745cb P aad8342212384388bab87b438d380ec7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "37946147a6bf4c67987556eaeaf55e2a" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 33339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:17:42.190371 14216 consensus_queue.cc:1035] T 132449bf61604c74993af0547ba745cb P aad8342212384388bab87b438d380ec7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "399de1a55cf8493b843174e3fc0f0d1f" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 41265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:17:42.201766 14219 mvcc.cc:205] Tried to move back new op lower bound from 7315974808314642432 to 7315974808038846464. Current Snapshot: MvccSnapshot[applied={T|T < 7315974808314642432}]
I20260807 18:17:42.215513 13534 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:36492:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:17:42.215744 13534 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:36492:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:17:42.219022 13534 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d02d0288512d4bcb8422f220de3c1c2a: Sending DeleteTablet for 3 replicas of tablet 132449bf61604c74993af0547ba745cb
I20260807 18:17:42.220228 13722 tablet_service.cc:1514] Processing DeleteTablet for tablet 132449bf61604c74993af0547ba745cb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:17:42 UTC) from {username='slave'} at 127.0.0.1:49394
I20260807 18:17:42.220535 13906 tablet_service.cc:1514] Processing DeleteTablet for tablet 132449bf61604c74993af0547ba745cb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:17:42 UTC) from {username='slave'} at 127.0.0.1:60524
I20260807 18:17:42.220943 14081 tablet_service.cc:1514] Processing DeleteTablet for tablet 132449bf61604c74993af0547ba745cb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:17:42 UTC) from {username='slave'} at 127.0.0.1:51498
I20260807 18:17:42.221915 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 13613
I20260807 18:17:42.228905 14235 tablet_replica.cc:330] T 132449bf61604c74993af0547ba745cb P 37946147a6bf4c67987556eaeaf55e2a: stopping tablet replica
I20260807 18:17:42.229451 14235 raft_consensus.cc:2229] T 132449bf61604c74993af0547ba745cb P 37946147a6bf4c67987556eaeaf55e2a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:17:42.230060 14235 raft_consensus.cc:2258] T 132449bf61604c74993af0547ba745cb P 37946147a6bf4c67987556eaeaf55e2a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:17:42.232487 14235 ts_tablet_manager.cc:1905] T 132449bf61604c74993af0547ba745cb P 37946147a6bf4c67987556eaeaf55e2a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:17:42.238519 14235 ts_tablet_manager.cc:1918] T 132449bf61604c74993af0547ba745cb P 37946147a6bf4c67987556eaeaf55e2a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:17:42.238723 14235 log.cc:1196] T 132449bf61604c74993af0547ba745cb P 37946147a6bf4c67987556eaeaf55e2a: Deleting WAL directory at /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126328808105-19048-0/minicluster-data/ts-1/wal/wals/132449bf61604c74993af0547ba745cb
I20260807 18:17:42.239367 14235 ts_tablet_manager.cc:1939] T 132449bf61604c74993af0547ba745cb P 37946147a6bf4c67987556eaeaf55e2a: Deleting consensus metadata
I20260807 18:17:42.239400 14236 tablet_replica.cc:330] T 132449bf61604c74993af0547ba745cb P aad8342212384388bab87b438d380ec7: stopping tablet replica
I20260807 18:17:42.241693 13513 catalog_manager.cc:4841] TS 37946147a6bf4c67987556eaeaf55e2a (127.18.154.2:33339): tablet 132449bf61604c74993af0547ba745cb (table test-table [id=89a13cfae26344578ab8c53c01e0b17c]) successfully deleted
I20260807 18:17:42.242396 14236 raft_consensus.cc:2229] T 132449bf61604c74993af0547ba745cb P aad8342212384388bab87b438d380ec7 [term 1 LEADER]: Raft consensus shutting down.
I20260807 18:17:42.242733 14197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:42.223563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49398 (local address 127.18.154.1:41265)
0807 18:17:42.223789 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:42.223799 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:42.228487 (+ 4688us) server_negotiation.cc:407] Connection header received
0807 18:17:42.228782 (+ 295us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:42.228788 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:42.228914 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:42.229028 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:17:42.229809 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:42.230970 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:42.233018 (+ 2048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:42.233670 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:42.233783 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:42.240452 (+ 6669us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:17:42.240468 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:17:42.240486 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:17:42.241346 (+ 860us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:17:42.241414 (+ 68us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:17:42.241443 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:17:42.241558 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:42.242173 (+ 615us) server_negotiation.cc:299] Negotiation successful
0807 18:17:42.242317 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260807 18:17:42.243242 14236 raft_consensus.cc:2258] T 132449bf61604c74993af0547ba745cb P aad8342212384388bab87b438d380ec7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:17:42.244132 14234 tablet_replica.cc:330] T 132449bf61604c74993af0547ba745cb P 399de1a55cf8493b843174e3fc0f0d1f: stopping tablet replica
I20260807 18:17:42.244290 13806 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:17:42.246342 14236 ts_tablet_manager.cc:1905] T 132449bf61604c74993af0547ba745cb P aad8342212384388bab87b438d380ec7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:17:42.252924 14236 ts_tablet_manager.cc:1918] T 132449bf61604c74993af0547ba745cb P aad8342212384388bab87b438d380ec7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:17:42.253130 14236 log.cc:1196] T 132449bf61604c74993af0547ba745cb P aad8342212384388bab87b438d380ec7: Deleting WAL directory at /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786126328808105-19048-0/minicluster-data/ts-2/wal/wals/132449bf61604c74993af0547ba745cb
I20260807 18:17:42.253742 14236 ts_tablet_manager.cc:1939] T 132449bf61604c74993af0547ba745cb P aad8342212384388bab87b438d380ec7: Deleting consensus metadata
I20260807 18:17:42.255317 13511 catalog_manager.cc:4841] TS aad8342212384388bab87b438d380ec7 (127.18.154.3:43361): tablet 132449bf61604c74993af0547ba745cb (table test-table [id=89a13cfae26344578ab8c53c01e0b17c]) successfully deleted
I20260807 18:17:42.244670 14234 raft_consensus.cc:2229] T 132449bf61604c74993af0547ba745cb P 399de1a55cf8493b843174e3fc0f0d1f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:17:42.365641 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 13613
I20260807 18:17:42.365612 14234 raft_consensus.cc:2258] T 132449bf61604c74993af0547ba745cb P 399de1a55cf8493b843174e3fc0f0d1f [term 1 FOLLOWER]: Raft consensus is shut down!
W20260807 18:17:42.389053 13511 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.154.1:41265 (error 108)
I20260807 18:17:42.389608 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 13821
I20260807 18:17:42.391537 14240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:42.390649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.1:41265 (local address 127.0.0.1:49406)
0807 18:17:42.391162 (+ 513us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:42.391310 (+ 148us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.1:41265: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":378,"thread_start_us":265,"threads_started":1}
W20260807 18:17:42.391858 13511 catalog_manager.cc:4584] TS 399de1a55cf8493b843174e3fc0f0d1f (127.18.154.1:41265): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 132449bf61604c74993af0547ba745cb: Network error: Client connection negotiation failed: client connection to 127.18.154.1:41265: connect: Connection refused (error 111)
I20260807 18:17:42.397740 14200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:42.390580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60528 (local address 127.18.154.2:33339)
0807 18:17:42.390781 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:42.390788 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:42.391146 (+ 358us) server_negotiation.cc:407] Connection header received
0807 18:17:42.391225 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:42.391229 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:42.391327 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:42.391440 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:17:42.391925 (+ 485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:42.392751 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:42.394414 (+ 1663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:42.394851 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:42.395007 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:42.396253 (+ 1246us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:17:42.396263 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:17:42.396278 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:17:42.396886 (+ 608us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:17:42.396934 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:17:42.396959 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:17:42.397089 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:42.397211 (+ 122us) server_negotiation.cc:299] Negotiation successful
0807 18:17:42.397325 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260807 18:17:42.398403 13988 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:17:42.445598 14240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:42.445012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.1:41265 (local address 127.0.0.1:49408)
0807 18:17:42.445275 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:42.445386 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.1:41265: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99}
I20260807 18:17:42.446841 14240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:42.446380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.1:41265 (local address 127.0.0.1:49420)
0807 18:17:42.446582 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:42.446681 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.1:41265: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260807 18:17:42.487857 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 13821
I20260807 18:17:42.508011 14240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:42.507327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.1:41265 (local address 127.0.0.1:49432)
0807 18:17:42.507644 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:42.507776 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.1:41265: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106}
I20260807 18:17:42.509568 14240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:42.508985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.1:41265 (local address 127.0.0.1:49440)
0807 18:17:42.509207 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:42.509319 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.1:41265: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260807 18:17:42.514798 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 13999
I20260807 18:17:42.522673 14203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:42.515608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51518 (local address 127.18.154.3:43361)
0807 18:17:42.516111 (+ 503us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:42.516116 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:42.516131 (+ 15us) server_negotiation.cc:407] Connection header received
0807 18:17:42.516187 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:42.516190 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:42.516261 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:42.516394 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:17:42.516807 (+ 413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:42.517691 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:42.519006 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:42.519432 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:42.519585 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:42.520722 (+ 1137us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:17:42.520731 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:17:42.520748 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:17:42.521862 (+ 1114us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:17:42.521915 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:17:42.521940 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:17:42.522089 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:42.522230 (+ 141us) server_negotiation.cc:299] Negotiation successful
0807 18:17:42.522346 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":392}
I20260807 18:17:42.523440 14172 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:17:42.591017 14240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:42.590483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.1:41265 (local address 127.0.0.1:49456)
0807 18:17:42.590758 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:42.590859 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.1:41265: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104}
I20260807 18:17:42.592185 14240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:42.591756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.1:41265 (local address 127.0.0.1:49472)
0807 18:17:42.591972 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:42.592045 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.1:41265: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106}
I20260807 18:17:42.647349 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 13999
I20260807 18:17:42.674893 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 13490
I20260807 18:17:42.676146 13572 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:17:42.786132 14240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:42.785461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.1:41265 (local address 127.0.0.1:49484)
0807 18:17:42.785746 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:42.785910 (+ 164us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.1:41265: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260807 18:17:42.786409 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 13490
2026-08-07T18:17:42Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (3247 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126328808105-19048-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:17:42 dist-test-slave-gkw7 krb5kdc[14247](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:17:42 dist-test-slave-gkw7 krb5kdc[14247](info): set up 2 sockets
Aug 07 18:17:42 dist-test-slave-gkw7 krb5kdc[14247](info): commencing operation
krb5kdc: starting...
W20260807 18:17:44.226114 19048 mini_kdc.cc:121] Time spent starting KDC: real 1.387s user 0.003s sys 0.039s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126328808105-19048-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126328808105-19048-0/krb5kdc/test-user.keytab.
Aug 07 18:17:44 dist-test-slave-gkw7 krb5kdc[14247](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126664, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:17:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:17:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.154.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.154.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.154.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.62.keytab.
Entry for principal kudu/127.18.154.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.62.keytab.
I20260807 18:17:44.553994 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126328808105-19048-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126328808105-19048-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126328808105-19048-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.154.62:35003
--webserver_interface=127.18.154.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.62.keytab
--principal=kudu/127.18.154.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.154.20:44215
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.154.62:35003
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126328808105-19048-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126328808105-19048-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126328808105-19048-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:17:44.916034 14265 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:17:44.916417 14265 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:17:44.916594 14265 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:17:44.923938 14265 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:17:44.924109 14265 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:17:44.924247 14265 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:17:44.924350 14265 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:17:44.924448 14265 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:17:44.934501 14265 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.154.20:44215
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126328808105-19048-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126328808105-19048-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.154.62:35003
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.62.keytab
--principal=kudu/127.18.154.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.154.62:35003
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.154.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126328808105-19048-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:17:44.935956 14265 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:17:44.937366 14265 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:17:44.950078 14272 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:17:44.950066 14276 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:17:44.950335 14273 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:17:44.950928 14265 server_base.cc:1034] running on GCE node
I20260807 18:17:44.951742 14265 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:17:44.953234 14265 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:17:44.954475 14265 hybrid_clock.cc:648] HybridClock initialized: now 1786126664954378 us; error 92 us; skew 500 ppm
I20260807 18:17:44.954839 14265 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:17:44 dist-test-slave-gkw7 krb5kdc[14247](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126664, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:17:44.958195 14265 init.cc:377] Logged in from keytab as kudu/127.18.154.62@KRBTEST.COM (short username kudu)
I20260807 18:17:44.959879 14265 webserver.cc:460] Webserver started at http://127.18.154.62:40713/ using document root <none> and password file <none>
I20260807 18:17:44.960666 14265 fs_manager.cc:359] Metadata directory not provided
I20260807 18:17:44.960821 14265 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:17:44.961236 14265 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:17:44.964499 14265 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126328808105-19048-0/minicluster-data/master-0/data/instance:
uuid: "e40d7eae3c7b4d83b6d2138d2980d3e6"
format_stamp: "Formatted at 2026-08-07 18:17:44 on dist-test-slave-gkw7"
server_key: "6750e06a102afdb436da3642a627c791"
server_key_iv: "eb00a7a1c4930e0a18573e245a17adaa"
server_key_version: "encryptionkey@0"
I20260807 18:17:44.965283 14265 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126328808105-19048-0/minicluster-data/master-0/wal/instance:
uuid: "e40d7eae3c7b4d83b6d2138d2980d3e6"
format_stamp: "Formatted at 2026-08-07 18:17:44 on dist-test-slave-gkw7"
server_key: "6750e06a102afdb436da3642a627c791"
server_key_iv: "eb00a7a1c4930e0a18573e245a17adaa"
server_key_version: "encryptionkey@0"
I20260807 18:17:44.970798 14265 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260807 18:17:44.974498 14287 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:17:44.975979 14265 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 18:17:44.976168 14265 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126328808105-19048-0/minicluster-data/master-0/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126328808105-19048-0/minicluster-data/master-0/wal
uuid: "e40d7eae3c7b4d83b6d2138d2980d3e6"
format_stamp: "Formatted at 2026-08-07 18:17:44 on dist-test-slave-gkw7"
server_key: "6750e06a102afdb436da3642a627c791"
server_key_iv: "eb00a7a1c4930e0a18573e245a17adaa"
server_key_version: "encryptionkey@0"
I20260807 18:17:44.976384 14265 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126328808105-19048-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126328808105-19048-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126328808105-19048-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:17:44.997646 14265 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:17:45.002344 14265 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:17:45.002656 14265 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:17:45.022953 14265 rpc_server.cc:307] RPC server started. Bound to: 127.18.154.62:35003
I20260807 18:17:45.022984 14359 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.154.62:35003 every 8 connection(s)
I20260807 18:17:45.024536 14265 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
I20260807 18:17:45.026624 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 14265
I20260807 18:17:45.026819 19048 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126328808105-19048-0/minicluster-data/master-0/wal/instance
I20260807 18:17:45.027303 19048 external_mini_cluster.cc:1428] Setting key 4d7aca403a00d79e1cf01c688c0dedbb
I20260807 18:17:45.029553 14360 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 07 18:17:45 dist-test-slave-gkw7 krb5kdc[14247](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126664, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.154.62@KRBTEST.COM
I20260807 18:17:45.045068 14360 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e40d7eae3c7b4d83b6d2138d2980d3e6: Bootstrap starting.
I20260807 18:17:45.050253 14360 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e40d7eae3c7b4d83b6d2138d2980d3e6: Neither blocks nor log segments found. Creating new log.
I20260807 18:17:45.051723 14360 log.cc:824] T 00000000000000000000000000000000 P e40d7eae3c7b4d83b6d2138d2980d3e6: Log is configured to *not* fsync() on all Append() calls
I20260807 18:17:45.055594 14364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:45.029333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55272 (local address 127.18.154.62:35003)
0807 18:17:45.030289 (+ 956us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:45.030306 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:45.030363 (+ 57us) server_negotiation.cc:407] Connection header received
0807 18:17:45.033053 (+ 2690us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:45.033088 (+ 35us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:45.033735 (+ 647us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:45.034270 (+ 535us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:17:45.035665 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:45.038106 (+ 2441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:45.039451 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:45.040662 (+ 1211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:45.040869 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:45.045221 (+ 4352us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:17:45.045236 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:17:45.045258 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:17:45.045311 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:17:45.049978 (+ 4667us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:17:45.050877 (+ 899us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:17:45.050887 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:17:45.050897 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:17:45.051147 (+ 250us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:17:45.051591 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:17:45.051596 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:17:45.051600 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:17:45.052503 (+ 903us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:17:45.052805 (+ 302us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:45.053431 (+ 626us) server_negotiation.cc:299] Negotiation successful
0807 18:17:45.053998 (+ 567us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":683,"thread_start_us":412,"threads_started":1}
I20260807 18:17:45.055927 14360 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e40d7eae3c7b4d83b6d2138d2980d3e6: No bootstrap required, opened a new log
I20260807 18:17:45.065492 14360 raft_consensus.cc:348] T 00000000000000000000000000000000 P e40d7eae3c7b4d83b6d2138d2980d3e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e40d7eae3c7b4d83b6d2138d2980d3e6" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 35003 } }
I20260807 18:17:45.065997 14360 raft_consensus.cc:374] T 00000000000000000000000000000000 P e40d7eae3c7b4d83b6d2138d2980d3e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:17:45.066097 14360 raft_consensus.cc:729] T 00000000000000000000000000000000 P e40d7eae3c7b4d83b6d2138d2980d3e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e40d7eae3c7b4d83b6d2138d2980d3e6, State: Initialized, Role: FOLLOWER
I20260807 18:17:45.066655 14360 consensus_queue.cc:260] T 00000000000000000000000000000000 P e40d7eae3c7b4d83b6d2138d2980d3e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e40d7eae3c7b4d83b6d2138d2980d3e6" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 35003 } }
I20260807 18:17:45.066872 14360 raft_consensus.cc:388] T 00000000000000000000000000000000 P e40d7eae3c7b4d83b6d2138d2980d3e6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:17:45.066983 14360 raft_consensus.cc:482] T 00000000000000000000000000000000 P e40d7eae3c7b4d83b6d2138d2980d3e6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:17:45.067134 14360 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e40d7eae3c7b4d83b6d2138d2980d3e6 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:17:45.069998 14360 raft_consensus.cc:504] T 00000000000000000000000000000000 P e40d7eae3c7b4d83b6d2138d2980d3e6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e40d7eae3c7b4d83b6d2138d2980d3e6" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 35003 } }
I20260807 18:17:45.070559 14360 leader_election.cc:302] T 00000000000000000000000000000000 P e40d7eae3c7b4d83b6d2138d2980d3e6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e40d7eae3c7b4d83b6d2138d2980d3e6; no voters:
I20260807 18:17:45.071163 14360 leader_election.cc:288] T 00000000000000000000000000000000 P e40d7eae3c7b4d83b6d2138d2980d3e6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:17:45.071321 14366 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e40d7eae3c7b4d83b6d2138d2980d3e6 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:17:45.072326 14366 raft_consensus.cc:686] T 00000000000000000000000000000000 P e40d7eae3c7b4d83b6d2138d2980d3e6 [term 1 LEADER]: Becoming Leader. State: Replica: e40d7eae3c7b4d83b6d2138d2980d3e6, State: Running, Role: LEADER
I20260807 18:17:45.073005 14366 consensus_queue.cc:237] T 00000000000000000000000000000000 P e40d7eae3c7b4d83b6d2138d2980d3e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e40d7eae3c7b4d83b6d2138d2980d3e6" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 35003 } }
I20260807 18:17:45.073907 14360 sys_catalog.cc:564] T 00000000000000000000000000000000 P e40d7eae3c7b4d83b6d2138d2980d3e6 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:17:45.075955 14367 sys_catalog.cc:455] T 00000000000000000000000000000000 P e40d7eae3c7b4d83b6d2138d2980d3e6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e40d7eae3c7b4d83b6d2138d2980d3e6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e40d7eae3c7b4d83b6d2138d2980d3e6" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 35003 } } }
I20260807 18:17:45.077078 14367 sys_catalog.cc:458] T 00000000000000000000000000000000 P e40d7eae3c7b4d83b6d2138d2980d3e6 [sys.catalog]: This master's current role is: LEADER
I20260807 18:17:45.077395 14368 sys_catalog.cc:455] T 00000000000000000000000000000000 P e40d7eae3c7b4d83b6d2138d2980d3e6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e40d7eae3c7b4d83b6d2138d2980d3e6. Latest consensus state: current_term: 1 leader_uuid: "e40d7eae3c7b4d83b6d2138d2980d3e6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e40d7eae3c7b4d83b6d2138d2980d3e6" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 35003 } } }
I20260807 18:17:45.077891 14368 sys_catalog.cc:458] T 00000000000000000000000000000000 P e40d7eae3c7b4d83b6d2138d2980d3e6 [sys.catalog]: This master's current role is: LEADER
I20260807 18:17:45.083160 14382 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:17:45.089754 14382 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:17:45.100596 14382 catalog_manager.cc:1347] Generated new cluster ID: 16f877912d5844b0a62cebd1f996b47f
I20260807 18:17:45.100747 14382 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:17:45.122572 14382 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:17:45.124574 14382 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:17:45.136193 14382 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e40d7eae3c7b4d83b6d2138d2980d3e6: Generated new TSK 0
I20260807 18:17:45.137395 14382 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.154.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.154.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.154.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.1.keytab.
Entry for principal kudu/127.18.154.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.1.keytab.
I20260807 18:17:45.215212 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126328808105-19048-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126328808105-19048-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126328808105-19048-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126328808105-19048-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.154.1:0
--local_ip_for_outbound_sockets=127.18.154.1
--webserver_interface=127.18.154.1
--webserver_port=0
--tserver_master_addrs=127.18.154.62:35003
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.1.keytab
--principal=kudu/127.18.154.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.154.20:44215
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126328808105-19048-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126328808105-19048-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126328808105-19048-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:17:45.576397 14399 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:17:45.576694 14399 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:17:45.576851 14399 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:17:45.584661 14399 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:17:45.584785 14399 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:17:45.584996 14399 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.154.1
I20260807 18:17:45.595114 14399 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.154.20:44215
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126328808105-19048-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126328808105-19048-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.1.keytab
--principal=kudu/127.18.154.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.154.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126328808105-19048-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.154.1
--webserver_port=0
--tserver_master_addrs=127.18.154.62:35003
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.154.1
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126328808105-19048-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:17:45.596560 14399 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:17:45.597970 14399 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:17:45.612112 14405 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:17:45.612015 14406 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:17:45.612083 14409 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:17:45.612988 14399 server_base.cc:1034] running on GCE node
I20260807 18:17:45.613673 14399 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:17:45.614655 14399 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:17:45.615872 14399 hybrid_clock.cc:648] HybridClock initialized: now 1786126665615805 us; error 69 us; skew 500 ppm
I20260807 18:17:45.616163 14399 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:17:45 dist-test-slave-gkw7 krb5kdc[14247](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126665, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:17:45.619590 14399 init.cc:377] Logged in from keytab as kudu/127.18.154.1@KRBTEST.COM (short username kudu)
I20260807 18:17:45.621492 14399 webserver.cc:460] Webserver started at http://127.18.154.1:35915/ using document root <none> and password file <none>
I20260807 18:17:45.622444 14399 fs_manager.cc:359] Metadata directory not provided
I20260807 18:17:45.622627 14399 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:17:45.623039 14399 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:17:45.626662 14399 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126328808105-19048-0/minicluster-data/ts-0/data/instance:
uuid: "c19e3e4025824f81a1a63fa420a30bc2"
format_stamp: "Formatted at 2026-08-07 18:17:45 on dist-test-slave-gkw7"
server_key: "654b976b0d56f3a3af01bdf6060f9a0f"
server_key_iv: "c1b54216dea18e0ee18f93c067b6f940"
server_key_version: "encryptionkey@0"
I20260807 18:17:45.627632 14399 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126328808105-19048-0/minicluster-data/ts-0/wal/instance:
uuid: "c19e3e4025824f81a1a63fa420a30bc2"
format_stamp: "Formatted at 2026-08-07 18:17:45 on dist-test-slave-gkw7"
server_key: "654b976b0d56f3a3af01bdf6060f9a0f"
server_key_iv: "c1b54216dea18e0ee18f93c067b6f940"
server_key_version: "encryptionkey@0"
I20260807 18:17:45.634616 14399 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.004s
I20260807 18:17:45.638857 14416 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:17:45.640388 14399 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.002s
I20260807 18:17:45.640568 14399 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126328808105-19048-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126328808105-19048-0/minicluster-data/ts-0/wal
uuid: "c19e3e4025824f81a1a63fa420a30bc2"
format_stamp: "Formatted at 2026-08-07 18:17:45 on dist-test-slave-gkw7"
server_key: "654b976b0d56f3a3af01bdf6060f9a0f"
server_key_iv: "c1b54216dea18e0ee18f93c067b6f940"
server_key_version: "encryptionkey@0"
I20260807 18:17:45.640776 14399 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126328808105-19048-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126328808105-19048-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126328808105-19048-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:17:45.661899 14399 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:17:45.666594 14399 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:17:45.666926 14399 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:17:45.668027 14399 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:17:45.669556 14399 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:17:45.669637 14399 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:17:45.669806 14399 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:17:45.669867 14399 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:17:45.705169 14399 rpc_server.cc:307] RPC server started. Bound to: 127.18.154.1:41381
I20260807 18:17:45.705272 14576 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.154.1:41381 every 8 connection(s)
I20260807 18:17:45.706727 14399 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126328808105-19048-0/minicluster-data/ts-0/data/info.pb
I20260807 18:17:45.710449 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 14399
I20260807 18:17:45.710635 19048 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126328808105-19048-0/minicluster-data/ts-0/wal/instance
I20260807 18:17:45.711117 19048 external_mini_cluster.cc:1428] Setting key 4f61bd41277cd989852b97dc2c25b025
Aug 07 18:17:45 dist-test-slave-gkw7 krb5kdc[14247](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126665, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.1@KRBTEST.COM for kudu/127.18.154.62@KRBTEST.COM
I20260807 18:17:45.730574 14583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:45.709760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.1:38177 (local address 127.18.154.62:35003)
0807 18:17:45.710289 (+ 529us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:45.710297 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:45.711694 (+ 1397us) server_negotiation.cc:407] Connection header received
0807 18:17:45.713944 (+ 2250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:45.713952 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:45.714053 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:45.714186 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:17:45.715830 (+ 1644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:45.717008 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:45.718791 (+ 1783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:45.719498 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:45.719652 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:45.723627 (+ 3975us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:17:45.723637 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:17:45.723645 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:17:45.723695 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:17:45.726989 (+ 3294us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:17:45.728052 (+ 1063us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:17:45.728056 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:17:45.728058 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:17:45.728194 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:17:45.728713 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:17:45.728717 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:17:45.728719 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:17:45.729019 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:17:45.729133 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:45.730108 (+ 975us) server_negotiation.cc:299] Negotiation successful
0807 18:17:45.730298 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":389,"thread_start_us":261,"threads_started":1}
I20260807 18:17:45.732263 14584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:45.710440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:35003 (local address 127.18.154.1:38177)
0807 18:17:45.711488 (+ 1048us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:45.711541 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0807 18:17:45.713653 (+ 2112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:17:45.714412 (+ 759us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:17:45.714421 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:17:45.715002 (+ 581us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:17:45.715642 (+ 640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:45.715661 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:45.717240 (+ 1579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:45.717245 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:45.718639 (+ 1394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:45.718652 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:45.719744 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:45.719750 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:45.719957 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:45.720715 (+ 758us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:17:45.720738 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:17:45.723361 (+ 2623us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:17:45.727162 (+ 3801us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:17:45.727173 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:17:45.727195 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:17:45.727910 (+ 715us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:17:45.728308 (+ 398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:17:45.728315 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:17:45.728322 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:17:45.728583 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:17:45.729158 (+ 575us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:17:45.729169 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:17:45.729682 (+ 513us) client_negotiation.cc:769] Sending connection context
0807 18:17:45.730003 (+ 321us) client_negotiation.cc:240] Negotiation successful
0807 18:17:45.730671 (+ 668us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":743,"thread_start_us":420,"threads_started":1}
I20260807 18:17:45.734084 14579 heartbeater.cc:347] Connected to a master server at 127.18.154.62:35003
I20260807 18:17:45.734440 14579 heartbeater.cc:464] Registering TS with master...
I20260807 18:17:45.735209 14579 heartbeater.cc:511] Master 127.18.154.62:35003 requested a full tablet report, sending...
I20260807 18:17:45.737407 14310 ts_manager.cc:194] Registered new tserver with Master: c19e3e4025824f81a1a63fa420a30bc2 (127.18.154.1:41381)
I20260807 18:17:45.739010 14310 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.154.1@KRBTEST.COM'} at 127.18.154.1:38177
WARNING: no policy specified for kudu/127.18.154.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.154.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.154.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.2.keytab.
Entry for principal kudu/127.18.154.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.2.keytab.
I20260807 18:17:45.785615 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126328808105-19048-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126328808105-19048-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126328808105-19048-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126328808105-19048-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.154.2:0
--local_ip_for_outbound_sockets=127.18.154.2
--webserver_interface=127.18.154.2
--webserver_port=0
--tserver_master_addrs=127.18.154.62:35003
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.2.keytab
--principal=kudu/127.18.154.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.154.20:44215
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126328808105-19048-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126328808105-19048-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126328808105-19048-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:17:46.145457 14587 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:17:46.145915 14587 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:17:46.146104 14587 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:17:46.153709 14587 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:17:46.153895 14587 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:17:46.154105 14587 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.154.2
I20260807 18:17:46.164553 14587 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.154.20:44215
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126328808105-19048-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126328808105-19048-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.2.keytab
--principal=kudu/127.18.154.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.154.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126328808105-19048-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.154.2
--webserver_port=0
--tserver_master_addrs=127.18.154.62:35003
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.154.2
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126328808105-19048-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:17:46.166095 14587 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:17:46.167524 14587 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:17:46.182660 14592 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:17:46.182735 14595 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:17:46.183260 14593 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:17:46.184381 14587 server_base.cc:1034] running on GCE node
I20260807 18:17:46.185048 14587 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:17:46.186178 14587 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:17:46.187428 14587 hybrid_clock.cc:648] HybridClock initialized: now 1786126666187382 us; error 61 us; skew 500 ppm
I20260807 18:17:46.187779 14587 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:17:46 dist-test-slave-gkw7 krb5kdc[14247](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126666, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:17:46.191279 14587 init.cc:377] Logged in from keytab as kudu/127.18.154.2@KRBTEST.COM (short username kudu)
I20260807 18:17:46.192971 14587 webserver.cc:460] Webserver started at http://127.18.154.2:46841/ using document root <none> and password file <none>
I20260807 18:17:46.193812 14587 fs_manager.cc:359] Metadata directory not provided
I20260807 18:17:46.193970 14587 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:17:46.194324 14587 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:17:46.197337 14587 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126328808105-19048-0/minicluster-data/ts-1/data/instance:
uuid: "bfd1a5297b3d4aa68865aeb498b5c9cf"
format_stamp: "Formatted at 2026-08-07 18:17:46 on dist-test-slave-gkw7"
server_key: "f71e1a34ad89500ade261981ec38c288"
server_key_iv: "f7c3a64d01d6dca761c7fdd7c3304d34"
server_key_version: "encryptionkey@0"
I20260807 18:17:46.198232 14587 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126328808105-19048-0/minicluster-data/ts-1/wal/instance:
uuid: "bfd1a5297b3d4aa68865aeb498b5c9cf"
format_stamp: "Formatted at 2026-08-07 18:17:46 on dist-test-slave-gkw7"
server_key: "f71e1a34ad89500ade261981ec38c288"
server_key_iv: "f7c3a64d01d6dca761c7fdd7c3304d34"
server_key_version: "encryptionkey@0"
I20260807 18:17:46.204599 14587 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260807 18:17:46.209682 14612 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:17:46.211329 14587 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260807 18:17:46.211568 14587 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126328808105-19048-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126328808105-19048-0/minicluster-data/ts-1/wal
uuid: "bfd1a5297b3d4aa68865aeb498b5c9cf"
format_stamp: "Formatted at 2026-08-07 18:17:46 on dist-test-slave-gkw7"
server_key: "f71e1a34ad89500ade261981ec38c288"
server_key_iv: "f7c3a64d01d6dca761c7fdd7c3304d34"
server_key_version: "encryptionkey@0"
I20260807 18:17:46.211822 14587 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126328808105-19048-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126328808105-19048-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126328808105-19048-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:17:46.229178 14587 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:17:46.234282 14587 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:17:46.234630 14587 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:17:46.235743 14587 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:17:46.237313 14587 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:17:46.237433 14587 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:17:46.237625 14587 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:17:46.237715 14587 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:17:46.276710 14587 rpc_server.cc:307] RPC server started. Bound to: 127.18.154.2:44527
I20260807 18:17:46.276768 14771 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.154.2:44527 every 8 connection(s)
I20260807 18:17:46.278281 14587 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126328808105-19048-0/minicluster-data/ts-1/data/info.pb
I20260807 18:17:46.280567 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 14587
I20260807 18:17:46.280797 19048 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126328808105-19048-0/minicluster-data/ts-1/wal/instance
I20260807 18:17:46.281422 19048 external_mini_cluster.cc:1428] Setting key dd34301e87a37a20f40c33abc612e8a2
Aug 07 18:17:46 dist-test-slave-gkw7 krb5kdc[14247](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126666, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.2@KRBTEST.COM for kudu/127.18.154.62@KRBTEST.COM
I20260807 18:17:46.303066 14775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:46.281234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.2:56419 (local address 127.18.154.62:35003)
0807 18:17:46.282631 (+ 1397us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:46.282638 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:46.282985 (+ 347us) server_negotiation.cc:407] Connection header received
0807 18:17:46.285102 (+ 2117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:46.285111 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:46.285229 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:46.285364 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:17:46.287080 (+ 1716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:46.288396 (+ 1316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:46.290154 (+ 1758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:46.290998 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:46.291178 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:46.295363 (+ 4185us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:17:46.295374 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:17:46.295382 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:17:46.295442 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:17:46.298975 (+ 3533us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:17:46.300169 (+ 1194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:17:46.300173 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:17:46.300176 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:17:46.300326 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:17:46.300918 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:17:46.300922 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:17:46.300925 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:17:46.301292 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:17:46.301419 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:46.302551 (+ 1132us) server_negotiation.cc:299] Negotiation successful
0807 18:17:46.302760 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1254,"thread_start_us":1137,"threads_started":1}
I20260807 18:17:46.304632 14776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:46.281922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:35003 (local address 127.18.154.2:56419)
0807 18:17:46.282778 (+ 856us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:46.282828 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0807 18:17:46.284808 (+ 1980us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:17:46.285682 (+ 874us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:17:46.285691 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:17:46.286241 (+ 550us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:17:46.286871 (+ 630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:46.286889 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:46.288604 (+ 1715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:46.288608 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:46.289997 (+ 1389us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:46.290010 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:46.291262 (+ 1252us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:46.291268 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:46.291484 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:46.292260 (+ 776us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:17:46.292285 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:17:46.295046 (+ 2761us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:17:46.299255 (+ 4209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:17:46.299269 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:17:46.299293 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:17:46.299991 (+ 698us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:17:46.300450 (+ 459us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:17:46.300458 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:17:46.300465 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:17:46.300746 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:17:46.301437 (+ 691us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:17:46.301448 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:17:46.302033 (+ 585us) client_negotiation.cc:769] Sending connection context
0807 18:17:46.302414 (+ 381us) client_negotiation.cc:240] Negotiation successful
0807 18:17:46.303027 (+ 613us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":513,"thread_start_us":524,"threads_started":1}
I20260807 18:17:46.306493 14772 heartbeater.cc:347] Connected to a master server at 127.18.154.62:35003
I20260807 18:17:46.306883 14772 heartbeater.cc:464] Registering TS with master...
I20260807 18:17:46.307705 14772 heartbeater.cc:511] Master 127.18.154.62:35003 requested a full tablet report, sending...
I20260807 18:17:46.309345 14310 ts_manager.cc:194] Registered new tserver with Master: bfd1a5297b3d4aa68865aeb498b5c9cf (127.18.154.2:44527)
I20260807 18:17:46.310277 14310 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.154.2@KRBTEST.COM'} at 127.18.154.2:56419
WARNING: no policy specified for kudu/127.18.154.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.154.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.154.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.3.keytab.
Entry for principal kudu/127.18.154.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.3.keytab.
I20260807 18:17:46.361502 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126328808105-19048-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126328808105-19048-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126328808105-19048-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126328808105-19048-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.154.3:0
--local_ip_for_outbound_sockets=127.18.154.3
--webserver_interface=127.18.154.3
--webserver_port=0
--tserver_master_addrs=127.18.154.62:35003
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.3.keytab
--principal=kudu/127.18.154.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.154.20:44215
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126328808105-19048-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126328808105-19048-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126328808105-19048-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:17:46.735183 14779 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:17:46.735479 14779 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:17:46.735651 14779 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260807 18:17:46.742426 14579 heartbeater.cc:503] Master 127.18.154.62:35003 was elected leader, sending a full tablet report...
W20260807 18:17:46.743407 14779 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:17:46.743554 14779 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:17:46.743731 14779 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.154.3
I20260807 18:17:46.754571 14779 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.154.20:44215
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126328808105-19048-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126328808105-19048-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.3.keytab
--principal=kudu/127.18.154.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.154.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126328808105-19048-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.154.3
--webserver_port=0
--tserver_master_addrs=127.18.154.62:35003
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.154.3
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126328808105-19048-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:17:46.755968 14779 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:17:46.757332 14779 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:17:46.771426 14785 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:17:46.771387 14786 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:17:46.771347 14788 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:17:46.772672 14779 server_base.cc:1034] running on GCE node
I20260807 18:17:46.773298 14779 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:17:46.774442 14779 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:17:46.775722 14779 hybrid_clock.cc:648] HybridClock initialized: now 1786126666775647 us; error 53 us; skew 500 ppm
I20260807 18:17:46.776185 14779 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:17:46 dist-test-slave-gkw7 krb5kdc[14247](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126666, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:17:46.780346 14779 init.cc:377] Logged in from keytab as kudu/127.18.154.3@KRBTEST.COM (short username kudu)
I20260807 18:17:46.782388 14779 webserver.cc:460] Webserver started at http://127.18.154.3:36683/ using document root <none> and password file <none>
I20260807 18:17:46.783278 14779 fs_manager.cc:359] Metadata directory not provided
I20260807 18:17:46.783453 14779 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:17:46.783857 14779 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:17:46.787405 14779 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126328808105-19048-0/minicluster-data/ts-2/data/instance:
uuid: "e19b02eadb8c49f7922095aca8e5a859"
format_stamp: "Formatted at 2026-08-07 18:17:46 on dist-test-slave-gkw7"
server_key: "c9bd3519010dbca449356688dbdb8621"
server_key_iv: "29f69212c21c07afd6254758875a97b2"
server_key_version: "encryptionkey@0"
I20260807 18:17:46.788359 14779 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126328808105-19048-0/minicluster-data/ts-2/wal/instance:
uuid: "e19b02eadb8c49f7922095aca8e5a859"
format_stamp: "Formatted at 2026-08-07 18:17:46 on dist-test-slave-gkw7"
server_key: "c9bd3519010dbca449356688dbdb8621"
server_key_iv: "29f69212c21c07afd6254758875a97b2"
server_key_version: "encryptionkey@0"
I20260807 18:17:46.794708 14779 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260807 18:17:46.798454 14799 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:17:46.800047 14779 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 18:17:46.800282 14779 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126328808105-19048-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126328808105-19048-0/minicluster-data/ts-2/wal
uuid: "e19b02eadb8c49f7922095aca8e5a859"
format_stamp: "Formatted at 2026-08-07 18:17:46 on dist-test-slave-gkw7"
server_key: "c9bd3519010dbca449356688dbdb8621"
server_key_iv: "29f69212c21c07afd6254758875a97b2"
server_key_version: "encryptionkey@0"
I20260807 18:17:46.800494 14779 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126328808105-19048-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126328808105-19048-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126328808105-19048-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:17:46.813901 14779 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:17:46.819336 14779 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:17:46.819739 14779 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:17:46.820896 14779 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:17:46.822794 14779 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:17:46.822968 14779 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:17:46.823165 14779 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:17:46.823256 14779 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:17:46.862921 14779 rpc_server.cc:307] RPC server started. Bound to: 127.18.154.3:37743
I20260807 18:17:46.862953 14962 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.154.3:37743 every 8 connection(s)
I20260807 18:17:46.864567 14779 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126328808105-19048-0/minicluster-data/ts-2/data/info.pb
I20260807 18:17:46.870797 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 14779
I20260807 18:17:46.871013 19048 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126328808105-19048-0/minicluster-data/ts-2/wal/instance
I20260807 18:17:46.871459 19048 external_mini_cluster.cc:1428] Setting key e3971f332b27968e631f4ca2f1f1ac0b
Aug 07 18:17:46 dist-test-slave-gkw7 krb5kdc[14247](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126666, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.3@KRBTEST.COM for kudu/127.18.154.62@KRBTEST.COM
I20260807 18:17:46.885978 14965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:46.867557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.3:48697 (local address 127.18.154.62:35003)
0807 18:17:46.868097 (+ 540us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:46.868101 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:46.868883 (+ 782us) server_negotiation.cc:407] Connection header received
0807 18:17:46.870894 (+ 2011us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:46.870899 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:46.870997 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:46.871120 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:17:46.872792 (+ 1672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:46.873796 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:46.875744 (+ 1948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:46.876142 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:46.876283 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:46.879811 (+ 3528us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:17:46.879817 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:17:46.879820 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:17:46.879852 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:17:46.882624 (+ 2772us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:17:46.883396 (+ 772us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:17:46.883403 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:17:46.883407 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:17:46.883535 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:17:46.883978 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:17:46.883985 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:17:46.883990 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:17:46.884281 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:17:46.884453 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:46.885266 (+ 813us) server_negotiation.cc:299] Negotiation successful
0807 18:17:46.885504 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":434,"thread_start_us":296,"threads_started":1}
I20260807 18:17:46.887177 14966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:46.867976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:35003 (local address 127.18.154.3:48697)
0807 18:17:46.868695 (+ 719us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:46.868743 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 18:17:46.870586 (+ 1843us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:17:46.871380 (+ 794us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:17:46.871389 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:17:46.871988 (+ 599us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:17:46.872601 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:46.872619 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:46.873952 (+ 1333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:46.873957 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:46.875573 (+ 1616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:46.875591 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:46.876383 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:46.876393 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:46.876643 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:46.877309 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:17:46.877333 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:17:46.879588 (+ 2255us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:17:46.882782 (+ 3194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:17:46.882791 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:17:46.882809 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:17:46.883256 (+ 447us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:17:46.883650 (+ 394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:17:46.883655 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:17:46.883659 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:17:46.883860 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:17:46.884413 (+ 553us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:17:46.884422 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:17:46.884781 (+ 359us) client_negotiation.cc:769] Sending connection context
0807 18:17:46.885092 (+ 311us) client_negotiation.cc:240] Negotiation successful
0807 18:17:46.885673 (+ 581us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":485,"thread_start_us":281,"threads_started":1}
I20260807 18:17:46.888639 14963 heartbeater.cc:347] Connected to a master server at 127.18.154.62:35003
I20260807 18:17:46.888962 14963 heartbeater.cc:464] Registering TS with master...
I20260807 18:17:46.889746 14963 heartbeater.cc:511] Master 127.18.154.62:35003 requested a full tablet report, sending...
I20260807 18:17:46.891160 14310 ts_manager.cc:194] Registered new tserver with Master: e19b02eadb8c49f7922095aca8e5a859 (127.18.154.3:37743)
I20260807 18:17:46.892012 14310 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.154.3@KRBTEST.COM'} at 127.18.154.3:48697
I20260807 18:17:46.899833 19048 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 18:17:46.917042 14965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:46.904067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48316 (local address 127.18.154.62:35003)
0807 18:17:46.904313 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:46.904323 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:46.904753 (+ 430us) server_negotiation.cc:407] Connection header received
0807 18:17:46.904951 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:46.904954 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:46.905019 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:46.905103 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:17:46.906811 (+ 1708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:46.907544 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:46.908890 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:46.909310 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:46.909407 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:46.911848 (+ 2441us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:17:46.911854 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:17:46.911857 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:17:46.911890 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:17:46.914392 (+ 2502us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:17:46.914984 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:17:46.914988 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:17:46.914990 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:17:46.915113 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:17:46.915519 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:17:46.915523 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:17:46.915525 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:17:46.915925 (+ 400us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:17:46.916112 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:46.916425 (+ 313us) server_negotiation.cc:299] Negotiation successful
0807 18:17:46.916645 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260807 18:17:46.921114 14310 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48316:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 18:17:46.928117 14310 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:17:46.961580 14980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:46.941110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.3:37743 (local address 127.0.0.1:43246)
0807 18:17:46.952053 (+ 10943us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:46.952082 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0807 18:17:46.952311 (+ 229us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:17:46.953039 (+ 728us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:17:46.953050 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:17:46.953098 (+ 48us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:17:46.954373 (+ 1275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:46.954386 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:46.956276 (+ 1890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:46.956281 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:46.958335 (+ 2054us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:46.958349 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:46.960348 (+ 1999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:46.960354 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:46.960608 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:46.960673 (+ 65us) client_negotiation.cc:769] Sending connection context
0807 18:17:46.960807 (+ 134us) client_negotiation.cc:240] Negotiation successful
0807 18:17:46.961025 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10732,"mutex_wait_us":25,"spinlock_wait_cycles":9344,"thread_start_us":10496,"threads_started":1}
I20260807 18:17:46.961907 14985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:46.942043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57682 (local address 127.18.154.1:41381)
0807 18:17:46.954042 (+ 11999us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:46.954053 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:46.954084 (+ 31us) server_negotiation.cc:407] Connection header received
0807 18:17:46.954291 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:46.954302 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:46.954592 (+ 290us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:46.954803 (+ 211us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:17:46.955332 (+ 529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:46.956930 (+ 1598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:46.959198 (+ 2268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:46.960419 (+ 1221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:46.960563 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:46.960828 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:46.961343 (+ 515us) server_negotiation.cc:299] Negotiation successful
0807 18:17:46.961519 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11822,"thread_start_us":11653,"threads_started":1}
I20260807 18:17:46.961673 14976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:46.940437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.2:44527 (local address 127.0.0.1:45086)
0807 18:17:46.941158 (+ 721us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:46.941236 (+ 78us) client_negotiation.cc:174] Beginning negotiation
0807 18:17:46.941417 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:17:46.953054 (+ 11637us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:17:46.953059 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:17:46.953117 (+ 58us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:17:46.953609 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:46.953626 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:46.955614 (+ 1988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:46.955620 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:46.958300 (+ 2680us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:46.958319 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:46.960325 (+ 2006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:46.960335 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:46.960967 (+ 632us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:46.960999 (+ 32us) client_negotiation.cc:769] Sending connection context
0807 18:17:46.961071 (+ 72us) client_negotiation.cc:240] Negotiation successful
0807 18:17:46.961206 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":477,"mutex_wait_us":362,"spinlock_wait_cycles":10624,"thread_start_us":320,"threads_started":1}
I20260807 18:17:46.961984 14982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:46.941586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.1:41381 (local address 127.0.0.1:57682)
0807 18:17:46.952879 (+ 11293us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:46.952895 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 18:17:46.954112 (+ 1217us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:17:46.954787 (+ 675us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:17:46.954791 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:17:46.954817 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:17:46.955178 (+ 361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:46.955187 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:46.957107 (+ 1920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:46.957113 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:46.958977 (+ 1864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:46.959031 (+ 54us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:46.960986 (+ 1955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:46.960993 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:46.961148 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:46.961174 (+ 26us) client_negotiation.cc:769] Sending connection context
0807 18:17:46.961290 (+ 116us) client_negotiation.cc:240] Negotiation successful
0807 18:17:46.961425 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11178,"spinlock_wait_cycles":896,"thread_start_us":11048,"threads_started":1}
I20260807 18:17:46.963268 14983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:46.941860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43246 (local address 127.18.154.3:37743)
0807 18:17:46.950664 (+ 8804us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:46.950674 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:46.952371 (+ 1697us) server_negotiation.cc:407] Connection header received
0807 18:17:46.952485 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:46.952493 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:46.952795 (+ 302us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:46.953038 (+ 243us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:17:46.954597 (+ 1559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:46.956091 (+ 1494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:46.958560 (+ 2469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:46.960049 (+ 1489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:46.962306 (+ 2257us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:46.962581 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:46.962666 (+ 85us) server_negotiation.cc:299] Negotiation successful
0807 18:17:46.962920 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8603,"thread_start_us":8430,"threads_started":1}
I20260807 18:17:46.963804 14979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:46.940654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45086 (local address 127.18.154.2:44527)
0807 18:17:46.952324 (+ 11670us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:46.952334 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:46.952360 (+ 26us) server_negotiation.cc:407] Connection header received
0807 18:17:46.952485 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:46.952493 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:46.952795 (+ 302us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:46.953262 (+ 467us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:17:46.953818 (+ 556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:46.955460 (+ 1642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:46.958484 (+ 3024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:46.960055 (+ 1571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:46.960606 (+ 551us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:46.960816 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:46.961856 (+ 1040us) server_negotiation.cc:299] Negotiation successful
0807 18:17:46.963455 (+ 1599us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11515,"thread_start_us":11403,"threads_started":1}
I20260807 18:17:46.966917 14481 tablet_service.cc:1467] Processing CreateTablet for tablet ff37fb2014f944bc90ef7fd8a6b0dbac (DEFAULT_TABLE table=test-table [id=d5560c32e41e4faf8acb0cc38d41ef86]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:17:46.967218 14865 tablet_service.cc:1467] Processing CreateTablet for tablet ff37fb2014f944bc90ef7fd8a6b0dbac (DEFAULT_TABLE table=test-table [id=d5560c32e41e4faf8acb0cc38d41ef86]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:17:46.968282 14481 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ff37fb2014f944bc90ef7fd8a6b0dbac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:17:46.968743 14865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ff37fb2014f944bc90ef7fd8a6b0dbac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:17:46.968272 14676 tablet_service.cc:1467] Processing CreateTablet for tablet ff37fb2014f944bc90ef7fd8a6b0dbac (DEFAULT_TABLE table=test-table [id=d5560c32e41e4faf8acb0cc38d41ef86]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:17:46.969645 14676 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ff37fb2014f944bc90ef7fd8a6b0dbac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:17:46.980463 14986 tablet_bootstrap.cc:492] T ff37fb2014f944bc90ef7fd8a6b0dbac P c19e3e4025824f81a1a63fa420a30bc2: Bootstrap starting.
I20260807 18:17:46.984481 14989 tablet_bootstrap.cc:492] T ff37fb2014f944bc90ef7fd8a6b0dbac P e19b02eadb8c49f7922095aca8e5a859: Bootstrap starting.
I20260807 18:17:46.985881 14986 tablet_bootstrap.cc:654] T ff37fb2014f944bc90ef7fd8a6b0dbac P c19e3e4025824f81a1a63fa420a30bc2: Neither blocks nor log segments found. Creating new log.
I20260807 18:17:46.986515 14990 tablet_bootstrap.cc:492] T ff37fb2014f944bc90ef7fd8a6b0dbac P bfd1a5297b3d4aa68865aeb498b5c9cf: Bootstrap starting.
I20260807 18:17:46.987560 14986 log.cc:824] T ff37fb2014f944bc90ef7fd8a6b0dbac P c19e3e4025824f81a1a63fa420a30bc2: Log is configured to *not* fsync() on all Append() calls
I20260807 18:17:46.988485 14989 tablet_bootstrap.cc:654] T ff37fb2014f944bc90ef7fd8a6b0dbac P e19b02eadb8c49f7922095aca8e5a859: Neither blocks nor log segments found. Creating new log.
I20260807 18:17:46.989725 14989 log.cc:824] T ff37fb2014f944bc90ef7fd8a6b0dbac P e19b02eadb8c49f7922095aca8e5a859: Log is configured to *not* fsync() on all Append() calls
I20260807 18:17:46.990024 14990 tablet_bootstrap.cc:654] T ff37fb2014f944bc90ef7fd8a6b0dbac P bfd1a5297b3d4aa68865aeb498b5c9cf: Neither blocks nor log segments found. Creating new log.
I20260807 18:17:46.991122 14986 tablet_bootstrap.cc:492] T ff37fb2014f944bc90ef7fd8a6b0dbac P c19e3e4025824f81a1a63fa420a30bc2: No bootstrap required, opened a new log
I20260807 18:17:46.991366 14990 log.cc:824] T ff37fb2014f944bc90ef7fd8a6b0dbac P bfd1a5297b3d4aa68865aeb498b5c9cf: Log is configured to *not* fsync() on all Append() calls
I20260807 18:17:46.991468 14986 ts_tablet_manager.cc:1397] T ff37fb2014f944bc90ef7fd8a6b0dbac P c19e3e4025824f81a1a63fa420a30bc2: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.003s
I20260807 18:17:46.993108 14989 tablet_bootstrap.cc:492] T ff37fb2014f944bc90ef7fd8a6b0dbac P e19b02eadb8c49f7922095aca8e5a859: No bootstrap required, opened a new log
I20260807 18:17:46.993414 14989 ts_tablet_manager.cc:1397] T ff37fb2014f944bc90ef7fd8a6b0dbac P e19b02eadb8c49f7922095aca8e5a859: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260807 18:17:46.994362 14990 tablet_bootstrap.cc:492] T ff37fb2014f944bc90ef7fd8a6b0dbac P bfd1a5297b3d4aa68865aeb498b5c9cf: No bootstrap required, opened a new log
I20260807 18:17:46.994628 14990 ts_tablet_manager.cc:1397] T ff37fb2014f944bc90ef7fd8a6b0dbac P bfd1a5297b3d4aa68865aeb498b5c9cf: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260807 18:17:47.002141 14990 raft_consensus.cc:348] T ff37fb2014f944bc90ef7fd8a6b0dbac P bfd1a5297b3d4aa68865aeb498b5c9cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfd1a5297b3d4aa68865aeb498b5c9cf" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 44527 } } peers { permanent_uuid: "c19e3e4025824f81a1a63fa420a30bc2" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 41381 } } peers { permanent_uuid: "e19b02eadb8c49f7922095aca8e5a859" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 37743 } }
I20260807 18:17:47.002666 14990 raft_consensus.cc:374] T ff37fb2014f944bc90ef7fd8a6b0dbac P bfd1a5297b3d4aa68865aeb498b5c9cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:17:47.002789 14990 raft_consensus.cc:729] T ff37fb2014f944bc90ef7fd8a6b0dbac P bfd1a5297b3d4aa68865aeb498b5c9cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bfd1a5297b3d4aa68865aeb498b5c9cf, State: Initialized, Role: FOLLOWER
I20260807 18:17:47.003618 14990 consensus_queue.cc:260] T ff37fb2014f944bc90ef7fd8a6b0dbac P bfd1a5297b3d4aa68865aeb498b5c9cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfd1a5297b3d4aa68865aeb498b5c9cf" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 44527 } } peers { permanent_uuid: "c19e3e4025824f81a1a63fa420a30bc2" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 41381 } } peers { permanent_uuid: "e19b02eadb8c49f7922095aca8e5a859" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 37743 } }
I20260807 18:17:47.004287 14986 raft_consensus.cc:348] T ff37fb2014f944bc90ef7fd8a6b0dbac P c19e3e4025824f81a1a63fa420a30bc2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfd1a5297b3d4aa68865aeb498b5c9cf" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 44527 } } peers { permanent_uuid: "c19e3e4025824f81a1a63fa420a30bc2" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 41381 } } peers { permanent_uuid: "e19b02eadb8c49f7922095aca8e5a859" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 37743 } }
I20260807 18:17:47.004504 14989 raft_consensus.cc:348] T ff37fb2014f944bc90ef7fd8a6b0dbac P e19b02eadb8c49f7922095aca8e5a859 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfd1a5297b3d4aa68865aeb498b5c9cf" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 44527 } } peers { permanent_uuid: "c19e3e4025824f81a1a63fa420a30bc2" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 41381 } } peers { permanent_uuid: "e19b02eadb8c49f7922095aca8e5a859" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 37743 } }
I20260807 18:17:47.004889 14986 raft_consensus.cc:374] T ff37fb2014f944bc90ef7fd8a6b0dbac P c19e3e4025824f81a1a63fa420a30bc2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:17:47.004918 14989 raft_consensus.cc:374] T ff37fb2014f944bc90ef7fd8a6b0dbac P e19b02eadb8c49f7922095aca8e5a859 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:17:47.005013 14989 raft_consensus.cc:729] T ff37fb2014f944bc90ef7fd8a6b0dbac P e19b02eadb8c49f7922095aca8e5a859 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e19b02eadb8c49f7922095aca8e5a859, State: Initialized, Role: FOLLOWER
I20260807 18:17:47.005014 14986 raft_consensus.cc:729] T ff37fb2014f944bc90ef7fd8a6b0dbac P c19e3e4025824f81a1a63fa420a30bc2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c19e3e4025824f81a1a63fa420a30bc2, State: Initialized, Role: FOLLOWER
I20260807 18:17:47.005400 14772 heartbeater.cc:503] Master 127.18.154.62:35003 was elected leader, sending a full tablet report...
I20260807 18:17:47.005674 14986 consensus_queue.cc:260] T ff37fb2014f944bc90ef7fd8a6b0dbac P c19e3e4025824f81a1a63fa420a30bc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfd1a5297b3d4aa68865aeb498b5c9cf" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 44527 } } peers { permanent_uuid: "c19e3e4025824f81a1a63fa420a30bc2" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 41381 } } peers { permanent_uuid: "e19b02eadb8c49f7922095aca8e5a859" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 37743 } }
I20260807 18:17:47.005677 14989 consensus_queue.cc:260] T ff37fb2014f944bc90ef7fd8a6b0dbac P e19b02eadb8c49f7922095aca8e5a859 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfd1a5297b3d4aa68865aeb498b5c9cf" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 44527 } } peers { permanent_uuid: "c19e3e4025824f81a1a63fa420a30bc2" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 41381 } } peers { permanent_uuid: "e19b02eadb8c49f7922095aca8e5a859" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 37743 } }
I20260807 18:17:47.007318 14963 heartbeater.cc:503] Master 127.18.154.62:35003 was elected leader, sending a full tablet report...
I20260807 18:17:47.007992 14989 ts_tablet_manager.cc:1428] T ff37fb2014f944bc90ef7fd8a6b0dbac P e19b02eadb8c49f7922095aca8e5a859: Time spent starting tablet: real 0.014s user 0.012s sys 0.004s
I20260807 18:17:47.009807 14986 ts_tablet_manager.cc:1428] T ff37fb2014f944bc90ef7fd8a6b0dbac P c19e3e4025824f81a1a63fa420a30bc2: Time spent starting tablet: real 0.018s user 0.010s sys 0.006s
I20260807 18:17:47.010787 14990 ts_tablet_manager.cc:1428] T ff37fb2014f944bc90ef7fd8a6b0dbac P bfd1a5297b3d4aa68865aeb498b5c9cf: Time spent starting tablet: real 0.016s user 0.007s sys 0.005s
W20260807 18:17:47.030520 14773 tablet.cc:2369] T ff37fb2014f944bc90ef7fd8a6b0dbac P bfd1a5297b3d4aa68865aeb498b5c9cf: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 18:17:47.116608 14964 tablet.cc:2369] T ff37fb2014f944bc90ef7fd8a6b0dbac P e19b02eadb8c49f7922095aca8e5a859: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:17:47.176903 14996 raft_consensus.cc:482] T ff37fb2014f944bc90ef7fd8a6b0dbac P bfd1a5297b3d4aa68865aeb498b5c9cf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:17:47.177295 14996 raft_consensus.cc:504] T ff37fb2014f944bc90ef7fd8a6b0dbac P bfd1a5297b3d4aa68865aeb498b5c9cf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfd1a5297b3d4aa68865aeb498b5c9cf" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 44527 } } peers { permanent_uuid: "c19e3e4025824f81a1a63fa420a30bc2" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 41381 } } peers { permanent_uuid: "e19b02eadb8c49f7922095aca8e5a859" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 37743 } }
I20260807 18:17:47.179255 14996 leader_election.cc:288] T ff37fb2014f944bc90ef7fd8a6b0dbac P bfd1a5297b3d4aa68865aeb498b5c9cf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c19e3e4025824f81a1a63fa420a30bc2 (127.18.154.1:41381), e19b02eadb8c49f7922095aca8e5a859 (127.18.154.3:37743)
I20260807 18:17:47.181244 14998 raft_consensus.cc:482] T ff37fb2014f944bc90ef7fd8a6b0dbac P e19b02eadb8c49f7922095aca8e5a859 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:17:47.181622 14998 raft_consensus.cc:504] T ff37fb2014f944bc90ef7fd8a6b0dbac P e19b02eadb8c49f7922095aca8e5a859 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfd1a5297b3d4aa68865aeb498b5c9cf" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 44527 } } peers { permanent_uuid: "c19e3e4025824f81a1a63fa420a30bc2" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 41381 } } peers { permanent_uuid: "e19b02eadb8c49f7922095aca8e5a859" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 37743 } }
I20260807 18:17:47.183781 14998 leader_election.cc:288] T ff37fb2014f944bc90ef7fd8a6b0dbac P e19b02eadb8c49f7922095aca8e5a859 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bfd1a5297b3d4aa68865aeb498b5c9cf (127.18.154.2:44527), c19e3e4025824f81a1a63fa420a30bc2 (127.18.154.1:41381)
I20260807 18:17:47.189631 14985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:47.179888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.2:55639 (local address 127.18.154.1:41381)
0807 18:17:47.180157 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:47.180165 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:47.180891 (+ 726us) server_negotiation.cc:407] Connection header received
0807 18:17:47.181191 (+ 300us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:47.181196 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:47.181317 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:47.181415 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:17:47.182221 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:47.183160 (+ 939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:47.186083 (+ 2923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:47.187971 (+ 1888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:47.188139 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:47.188259 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:47.189012 (+ 753us) server_negotiation.cc:299] Negotiation successful
0807 18:17:47.189216 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260807 18:17:47.190759 14999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:47.179656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.1:41381 (local address 127.18.154.2:55639)
0807 18:17:47.180758 (+ 1102us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:47.180786 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0807 18:17:47.180991 (+ 205us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:17:47.181438 (+ 447us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:17:47.181443 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:17:47.181470 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:17:47.182032 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:47.182046 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:47.183300 (+ 1254us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:47.183307 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:47.185824 (+ 2517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:47.185845 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:47.188508 (+ 2663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:47.188517 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:47.188764 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:47.188818 (+ 54us) client_negotiation.cc:769] Sending connection context
0807 18:17:47.189943 (+ 1125us) client_negotiation.cc:240] Negotiation successful
0807 18:17:47.190160 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":950,"mutex_wait_us":31,"thread_start_us":552,"threads_started":1}
I20260807 18:17:47.194123 14966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:47.184100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.2:44527 (local address 127.18.154.3:47421)
0807 18:17:47.184298 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:47.184316 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 18:17:47.184461 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:17:47.185324 (+ 863us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:17:47.185329 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:17:47.185359 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:17:47.185661 (+ 302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:47.185676 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:47.187513 (+ 1837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:47.187522 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:47.190595 (+ 3073us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:47.190626 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:47.192995 (+ 2369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:47.193003 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:47.193296 (+ 293us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:47.193354 (+ 58us) client_negotiation.cc:769] Sending connection context
0807 18:17:47.193477 (+ 123us) client_negotiation.cc:240] Negotiation successful
0807 18:17:47.193717 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":74}
I20260807 18:17:47.194124 14979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:47.184076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.3:47421 (local address 127.18.154.2:44527)
0807 18:17:47.184245 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:47.184254 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:47.184394 (+ 140us) server_negotiation.cc:407] Connection header received
0807 18:17:47.184587 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:47.184594 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:47.184666 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:47.184748 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:17:47.185837 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:47.187308 (+ 1471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:47.190850 (+ 3542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:47.192469 (+ 1619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:47.192679 (+ 210us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:47.192784 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:47.193564 (+ 780us) server_negotiation.cc:299] Negotiation successful
0807 18:17:47.193758 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":20,"server-negotiator.queue_time_us":60}
I20260807 18:17:47.197171 14707 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff37fb2014f944bc90ef7fd8a6b0dbac" candidate_uuid: "e19b02eadb8c49f7922095aca8e5a859" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfd1a5297b3d4aa68865aeb498b5c9cf" is_pre_election: true
I20260807 18:17:47.197728 14707 raft_consensus.cc:2454] T ff37fb2014f944bc90ef7fd8a6b0dbac P bfd1a5297b3d4aa68865aeb498b5c9cf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e19b02eadb8c49f7922095aca8e5a859 in term 0.
I20260807 18:17:47.198100 14511 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff37fb2014f944bc90ef7fd8a6b0dbac" candidate_uuid: "bfd1a5297b3d4aa68865aeb498b5c9cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c19e3e4025824f81a1a63fa420a30bc2" is_pre_election: true
I20260807 18:17:47.198733 14511 raft_consensus.cc:2454] T ff37fb2014f944bc90ef7fd8a6b0dbac P c19e3e4025824f81a1a63fa420a30bc2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bfd1a5297b3d4aa68865aeb498b5c9cf in term 0.
I20260807 18:17:47.199329 14983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:47.179735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.2:32947 (local address 127.18.154.3:37743)
0807 18:17:47.179970 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:47.179978 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:47.181084 (+ 1106us) server_negotiation.cc:407] Connection header received
0807 18:17:47.181302 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:47.181307 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:47.181392 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:47.181475 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:17:47.182418 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:47.183548 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:47.196928 (+ 13380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:47.198222 (+ 1294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:47.198388 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:47.198462 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:47.198895 (+ 433us) server_negotiation.cc:299] Negotiation successful
0807 18:17:47.199047 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260807 18:17:47.199489 15001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:47.179814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.3:37743 (local address 127.18.154.2:32947)
0807 18:17:47.180964 (+ 1150us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:47.180983 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 18:17:47.181145 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:17:47.181671 (+ 526us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:17:47.181675 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:17:47.181705 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:17:47.182261 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:47.182277 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:47.184770 (+ 2493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:47.184775 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:47.196721 (+ 11946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:47.196743 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:47.198507 (+ 1764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:47.198518 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:47.198723 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:47.198765 (+ 42us) client_negotiation.cc:769] Sending connection context
0807 18:17:47.198877 (+ 112us) client_negotiation.cc:240] Negotiation successful
0807 18:17:47.199123 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1007,"mutex_wait_us":5,"thread_start_us":727,"threads_started":1}
I20260807 18:17:47.199605 14800 leader_election.cc:302] T ff37fb2014f944bc90ef7fd8a6b0dbac P e19b02eadb8c49f7922095aca8e5a859 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bfd1a5297b3d4aa68865aeb498b5c9cf, e19b02eadb8c49f7922095aca8e5a859; no voters:
I20260807 18:17:47.200044 14998 raft_consensus.cc:2781] T ff37fb2014f944bc90ef7fd8a6b0dbac P e19b02eadb8c49f7922095aca8e5a859 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:17:47.200071 14613 leader_election.cc:302] T ff37fb2014f944bc90ef7fd8a6b0dbac P bfd1a5297b3d4aa68865aeb498b5c9cf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bfd1a5297b3d4aa68865aeb498b5c9cf, c19e3e4025824f81a1a63fa420a30bc2; no voters:
I20260807 18:17:47.200205 14998 raft_consensus.cc:482] T ff37fb2014f944bc90ef7fd8a6b0dbac P e19b02eadb8c49f7922095aca8e5a859 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:17:47.200338 14998 raft_consensus.cc:3037] T ff37fb2014f944bc90ef7fd8a6b0dbac P e19b02eadb8c49f7922095aca8e5a859 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:17:47.200368 14893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff37fb2014f944bc90ef7fd8a6b0dbac" candidate_uuid: "bfd1a5297b3d4aa68865aeb498b5c9cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e19b02eadb8c49f7922095aca8e5a859" is_pre_election: true
I20260807 18:17:47.200682 14996 raft_consensus.cc:2781] T ff37fb2014f944bc90ef7fd8a6b0dbac P bfd1a5297b3d4aa68865aeb498b5c9cf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:17:47.200824 14996 raft_consensus.cc:482] T ff37fb2014f944bc90ef7fd8a6b0dbac P bfd1a5297b3d4aa68865aeb498b5c9cf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:17:47.200901 14996 raft_consensus.cc:3037] T ff37fb2014f944bc90ef7fd8a6b0dbac P bfd1a5297b3d4aa68865aeb498b5c9cf [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:17:47.203022 14966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:47.184505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.1:41381 (local address 127.18.154.3:50091)
0807 18:17:47.194751 (+ 10246us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:47.194777 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0807 18:17:47.194929 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:17:47.195530 (+ 601us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:17:47.195539 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:17:47.195580 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:17:47.195839 (+ 259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:47.195853 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:47.198278 (+ 2425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:47.198286 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:47.200167 (+ 1881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:47.200186 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:47.202186 (+ 2000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:47.202196 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:47.202424 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:47.202464 (+ 40us) client_negotiation.cc:769] Sending connection context
0807 18:17:47.202568 (+ 104us) client_negotiation.cc:240] Negotiation successful
0807 18:17:47.202707 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10097,"thread_start_us":11557,"threads_started":1}
I20260807 18:17:47.203099 14985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:47.184991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.3:50091 (local address 127.18.154.1:41381)
0807 18:17:47.189887 (+ 4896us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:47.189895 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:47.194884 (+ 4989us) server_negotiation.cc:407] Connection header received
0807 18:17:47.195100 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:47.195107 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:47.195209 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:47.195319 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:17:47.196038 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:47.198137 (+ 2099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:47.200386 (+ 2249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:47.201755 (+ 1369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:47.201915 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:47.202014 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:47.202605 (+ 591us) server_negotiation.cc:299] Negotiation successful
0807 18:17:47.202801 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4799,"thread_start_us":12108,"threads_started":1}
I20260807 18:17:47.203671 14998 raft_consensus.cc:504] T ff37fb2014f944bc90ef7fd8a6b0dbac P e19b02eadb8c49f7922095aca8e5a859 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfd1a5297b3d4aa68865aeb498b5c9cf" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 44527 } } peers { permanent_uuid: "c19e3e4025824f81a1a63fa420a30bc2" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 41381 } } peers { permanent_uuid: "e19b02eadb8c49f7922095aca8e5a859" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 37743 } }
I20260807 18:17:47.203749 14996 raft_consensus.cc:504] T ff37fb2014f944bc90ef7fd8a6b0dbac P bfd1a5297b3d4aa68865aeb498b5c9cf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfd1a5297b3d4aa68865aeb498b5c9cf" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 44527 } } peers { permanent_uuid: "c19e3e4025824f81a1a63fa420a30bc2" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 41381 } } peers { permanent_uuid: "e19b02eadb8c49f7922095aca8e5a859" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 37743 } }
I20260807 18:17:47.204167 14893 raft_consensus.cc:2379] T ff37fb2014f944bc90ef7fd8a6b0dbac P e19b02eadb8c49f7922095aca8e5a859 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bfd1a5297b3d4aa68865aeb498b5c9cf in current term 1: Already voted for candidate e19b02eadb8c49f7922095aca8e5a859 in this term.
I20260807 18:17:47.204306 14998 leader_election.cc:288] T ff37fb2014f944bc90ef7fd8a6b0dbac P e19b02eadb8c49f7922095aca8e5a859 [CANDIDATE]: Term 1 election: Requested vote from peers bfd1a5297b3d4aa68865aeb498b5c9cf (127.18.154.2:44527), c19e3e4025824f81a1a63fa420a30bc2 (127.18.154.1:41381)
I20260807 18:17:47.204419 14996 leader_election.cc:288] T ff37fb2014f944bc90ef7fd8a6b0dbac P bfd1a5297b3d4aa68865aeb498b5c9cf [CANDIDATE]: Term 1 election: Requested vote from peers c19e3e4025824f81a1a63fa420a30bc2 (127.18.154.1:41381), e19b02eadb8c49f7922095aca8e5a859 (127.18.154.3:37743)
I20260807 18:17:47.205156 14511 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff37fb2014f944bc90ef7fd8a6b0dbac" candidate_uuid: "e19b02eadb8c49f7922095aca8e5a859" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c19e3e4025824f81a1a63fa420a30bc2" is_pre_election: true
I20260807 18:17:47.205312 14707 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff37fb2014f944bc90ef7fd8a6b0dbac" candidate_uuid: "e19b02eadb8c49f7922095aca8e5a859" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfd1a5297b3d4aa68865aeb498b5c9cf"
I20260807 18:17:47.205395 14511 raft_consensus.cc:2454] T ff37fb2014f944bc90ef7fd8a6b0dbac P c19e3e4025824f81a1a63fa420a30bc2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e19b02eadb8c49f7922095aca8e5a859 in term 0.
I20260807 18:17:47.205637 14707 raft_consensus.cc:2379] T ff37fb2014f944bc90ef7fd8a6b0dbac P bfd1a5297b3d4aa68865aeb498b5c9cf [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate e19b02eadb8c49f7922095aca8e5a859 in current term 1: Already voted for candidate bfd1a5297b3d4aa68865aeb498b5c9cf in this term.
I20260807 18:17:47.205833 14508 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff37fb2014f944bc90ef7fd8a6b0dbac" candidate_uuid: "bfd1a5297b3d4aa68865aeb498b5c9cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c19e3e4025824f81a1a63fa420a30bc2"
I20260807 18:17:47.206204 14508 raft_consensus.cc:3037] T ff37fb2014f944bc90ef7fd8a6b0dbac P c19e3e4025824f81a1a63fa420a30bc2 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:17:47.206702 14893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff37fb2014f944bc90ef7fd8a6b0dbac" candidate_uuid: "bfd1a5297b3d4aa68865aeb498b5c9cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e19b02eadb8c49f7922095aca8e5a859"
I20260807 18:17:47.205894 14509 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff37fb2014f944bc90ef7fd8a6b0dbac" candidate_uuid: "e19b02eadb8c49f7922095aca8e5a859" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c19e3e4025824f81a1a63fa420a30bc2"
I20260807 18:17:47.206943 14893 raft_consensus.cc:2379] T ff37fb2014f944bc90ef7fd8a6b0dbac P e19b02eadb8c49f7922095aca8e5a859 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate bfd1a5297b3d4aa68865aeb498b5c9cf in current term 1: Already voted for candidate e19b02eadb8c49f7922095aca8e5a859 in this term.
I20260807 18:17:47.207791 14800 leader_election.cc:302] T ff37fb2014f944bc90ef7fd8a6b0dbac P e19b02eadb8c49f7922095aca8e5a859 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e19b02eadb8c49f7922095aca8e5a859; no voters: bfd1a5297b3d4aa68865aeb498b5c9cf, c19e3e4025824f81a1a63fa420a30bc2
I20260807 18:17:47.208166 14998 raft_consensus.cc:2726] T ff37fb2014f944bc90ef7fd8a6b0dbac P e19b02eadb8c49f7922095aca8e5a859 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
W20260807 18:17:47.209571 14581 tablet.cc:2369] T ff37fb2014f944bc90ef7fd8a6b0dbac P c19e3e4025824f81a1a63fa420a30bc2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:17:47.209944 14508 raft_consensus.cc:2454] T ff37fb2014f944bc90ef7fd8a6b0dbac P c19e3e4025824f81a1a63fa420a30bc2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bfd1a5297b3d4aa68865aeb498b5c9cf in term 1.
I20260807 18:17:47.210399 14613 leader_election.cc:302] T ff37fb2014f944bc90ef7fd8a6b0dbac P bfd1a5297b3d4aa68865aeb498b5c9cf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bfd1a5297b3d4aa68865aeb498b5c9cf, c19e3e4025824f81a1a63fa420a30bc2; no voters: e19b02eadb8c49f7922095aca8e5a859
I20260807 18:17:47.210786 14996 raft_consensus.cc:2781] T ff37fb2014f944bc90ef7fd8a6b0dbac P bfd1a5297b3d4aa68865aeb498b5c9cf [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:17:47.211267 14996 raft_consensus.cc:686] T ff37fb2014f944bc90ef7fd8a6b0dbac P bfd1a5297b3d4aa68865aeb498b5c9cf [term 1 LEADER]: Becoming Leader. State: Replica: bfd1a5297b3d4aa68865aeb498b5c9cf, State: Running, Role: LEADER
I20260807 18:17:47.211840 14996 consensus_queue.cc:237] T ff37fb2014f944bc90ef7fd8a6b0dbac P bfd1a5297b3d4aa68865aeb498b5c9cf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfd1a5297b3d4aa68865aeb498b5c9cf" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 44527 } } peers { permanent_uuid: "c19e3e4025824f81a1a63fa420a30bc2" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 41381 } } peers { permanent_uuid: "e19b02eadb8c49f7922095aca8e5a859" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 37743 } }
I20260807 18:17:47.217012 14307 catalog_manager.cc:5495] T ff37fb2014f944bc90ef7fd8a6b0dbac P bfd1a5297b3d4aa68865aeb498b5c9cf reported cstate change: term changed from 0 to 1, leader changed from <none> to bfd1a5297b3d4aa68865aeb498b5c9cf (127.18.154.2). New cstate: current_term: 1 leader_uuid: "bfd1a5297b3d4aa68865aeb498b5c9cf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfd1a5297b3d4aa68865aeb498b5c9cf" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 44527 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c19e3e4025824f81a1a63fa420a30bc2" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 41381 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e19b02eadb8c49f7922095aca8e5a859" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 37743 } health_report { overall_health: UNKNOWN } } }
I20260807 18:17:47.263098 14979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:47.257333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45096 (local address 127.18.154.2:44527)
0807 18:17:47.257514 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:47.257519 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:47.257587 (+ 68us) server_negotiation.cc:407] Connection header received
0807 18:17:47.257811 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:47.257815 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:47.257912 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:47.258046 (+ 134us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:17:47.258419 (+ 373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:47.259305 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:47.260746 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:47.261162 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:47.261305 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:47.261912 (+ 607us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:17:47.262452 (+ 540us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:17:47.262622 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:47.262704 (+ 82us) server_negotiation.cc:299] Negotiation successful
0807 18:17:47.262791 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 18:17:47.272357 14508 raft_consensus.cc:1261] T ff37fb2014f944bc90ef7fd8a6b0dbac P c19e3e4025824f81a1a63fa420a30bc2 [term 1 FOLLOWER]: Refusing update from remote peer bfd1a5297b3d4aa68865aeb498b5c9cf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:17:47.272375 14893 raft_consensus.cc:1261] T ff37fb2014f944bc90ef7fd8a6b0dbac P e19b02eadb8c49f7922095aca8e5a859 [term 1 FOLLOWER]: Refusing update from remote peer bfd1a5297b3d4aa68865aeb498b5c9cf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:17:47.273581 15004 consensus_queue.cc:1035] T ff37fb2014f944bc90ef7fd8a6b0dbac P bfd1a5297b3d4aa68865aeb498b5c9cf [LEADER]: Connected to new peer: Peer: permanent_uuid: "c19e3e4025824f81a1a63fa420a30bc2" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 41381 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:17:47.274106 14996 consensus_queue.cc:1035] T ff37fb2014f944bc90ef7fd8a6b0dbac P bfd1a5297b3d4aa68865aeb498b5c9cf [LEADER]: Connected to new peer: Peer: permanent_uuid: "e19b02eadb8c49f7922095aca8e5a859" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 37743 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:17:47.295758 15009 mvcc.cc:205] Tried to move back new op lower bound from 7315974829135466496 to 7315974828903608320. Current Snapshot: MvccSnapshot[applied={T|T < 7315974829135466496}]
I20260807 18:17:47.324345 14307 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48316:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:17:47.324676 14307 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48316:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:17:47.329174 14307 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e40d7eae3c7b4d83b6d2138d2980d3e6: Sending DeleteTablet for 3 replicas of tablet ff37fb2014f944bc90ef7fd8a6b0dbac
I20260807 18:17:47.330901 14865 tablet_service.cc:1514] Processing DeleteTablet for tablet ff37fb2014f944bc90ef7fd8a6b0dbac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:17:47 UTC) from {username='kudu', principal='kudu/127.18.154.62@KRBTEST.COM'} at 127.0.0.1:43246
I20260807 18:17:47.330906 14676 tablet_service.cc:1514] Processing DeleteTablet for tablet ff37fb2014f944bc90ef7fd8a6b0dbac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:17:47 UTC) from {username='kudu', principal='kudu/127.18.154.62@KRBTEST.COM'} at 127.0.0.1:45086
I20260807 18:17:47.332211 14481 tablet_service.cc:1514] Processing DeleteTablet for tablet ff37fb2014f944bc90ef7fd8a6b0dbac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:17:47 UTC) from {username='kudu', principal='kudu/127.18.154.62@KRBTEST.COM'} at 127.0.0.1:57682
I20260807 18:17:47.332976 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 14399
I20260807 18:17:47.342857 15030 tablet_replica.cc:330] T ff37fb2014f944bc90ef7fd8a6b0dbac P c19e3e4025824f81a1a63fa420a30bc2: stopping tablet replica
I20260807 18:17:47.344803 15030 raft_consensus.cc:2229] T ff37fb2014f944bc90ef7fd8a6b0dbac P c19e3e4025824f81a1a63fa420a30bc2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:17:47.345978 15030 raft_consensus.cc:2258] T ff37fb2014f944bc90ef7fd8a6b0dbac P c19e3e4025824f81a1a63fa420a30bc2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:17:47.346800 15029 tablet_replica.cc:330] T ff37fb2014f944bc90ef7fd8a6b0dbac P e19b02eadb8c49f7922095aca8e5a859: stopping tablet replica
I20260807 18:17:47.347290 15029 raft_consensus.cc:2229] T ff37fb2014f944bc90ef7fd8a6b0dbac P e19b02eadb8c49f7922095aca8e5a859 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:17:47.347909 15029 raft_consensus.cc:2258] T ff37fb2014f944bc90ef7fd8a6b0dbac P e19b02eadb8c49f7922095aca8e5a859 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:17:47.348805 15028 tablet_replica.cc:330] T ff37fb2014f944bc90ef7fd8a6b0dbac P bfd1a5297b3d4aa68865aeb498b5c9cf: stopping tablet replica
I20260807 18:17:47.349243 15028 raft_consensus.cc:2229] T ff37fb2014f944bc90ef7fd8a6b0dbac P bfd1a5297b3d4aa68865aeb498b5c9cf [term 1 LEADER]: Raft consensus shutting down.
I20260807 18:17:47.350266 15030 ts_tablet_manager.cc:1905] T ff37fb2014f944bc90ef7fd8a6b0dbac P c19e3e4025824f81a1a63fa420a30bc2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:17:47.350703 15028 raft_consensus.cc:2258] T ff37fb2014f944bc90ef7fd8a6b0dbac P bfd1a5297b3d4aa68865aeb498b5c9cf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:17:47.351671 15029 ts_tablet_manager.cc:1905] T ff37fb2014f944bc90ef7fd8a6b0dbac P e19b02eadb8c49f7922095aca8e5a859: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:17:47.352680 15028 ts_tablet_manager.cc:1905] T ff37fb2014f944bc90ef7fd8a6b0dbac P bfd1a5297b3d4aa68865aeb498b5c9cf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:17:47.358096 15030 ts_tablet_manager.cc:1918] T ff37fb2014f944bc90ef7fd8a6b0dbac P c19e3e4025824f81a1a63fa420a30bc2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:17:47.358332 15030 log.cc:1196] T ff37fb2014f944bc90ef7fd8a6b0dbac P c19e3e4025824f81a1a63fa420a30bc2: Deleting WAL directory at /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126328808105-19048-0/minicluster-data/ts-0/wal/wals/ff37fb2014f944bc90ef7fd8a6b0dbac
I20260807 18:17:47.358974 15030 ts_tablet_manager.cc:1939] T ff37fb2014f944bc90ef7fd8a6b0dbac P c19e3e4025824f81a1a63fa420a30bc2: Deleting consensus metadata
I20260807 18:17:47.359081 15029 ts_tablet_manager.cc:1918] T ff37fb2014f944bc90ef7fd8a6b0dbac P e19b02eadb8c49f7922095aca8e5a859: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:17:47.359259 15029 log.cc:1196] T ff37fb2014f944bc90ef7fd8a6b0dbac P e19b02eadb8c49f7922095aca8e5a859: Deleting WAL directory at /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126328808105-19048-0/minicluster-data/ts-2/wal/wals/ff37fb2014f944bc90ef7fd8a6b0dbac
I20260807 18:17:47.360335 15028 ts_tablet_manager.cc:1918] T ff37fb2014f944bc90ef7fd8a6b0dbac P bfd1a5297b3d4aa68865aeb498b5c9cf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:17:47.360520 15028 log.cc:1196] T ff37fb2014f944bc90ef7fd8a6b0dbac P bfd1a5297b3d4aa68865aeb498b5c9cf: Deleting WAL directory at /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786126328808105-19048-0/minicluster-data/ts-1/wal/wals/ff37fb2014f944bc90ef7fd8a6b0dbac
I20260807 18:17:47.361404 14288 catalog_manager.cc:4841] TS c19e3e4025824f81a1a63fa420a30bc2 (127.18.154.1:41381): tablet ff37fb2014f944bc90ef7fd8a6b0dbac (table test-table [id=d5560c32e41e4faf8acb0cc38d41ef86]) successfully deleted
I20260807 18:17:47.361804 15028 ts_tablet_manager.cc:1939] T ff37fb2014f944bc90ef7fd8a6b0dbac P bfd1a5297b3d4aa68865aeb498b5c9cf: Deleting consensus metadata
I20260807 18:17:47.363082 15029 ts_tablet_manager.cc:1939] T ff37fb2014f944bc90ef7fd8a6b0dbac P e19b02eadb8c49f7922095aca8e5a859: Deleting consensus metadata
I20260807 18:17:47.363319 14288 catalog_manager.cc:4841] TS bfd1a5297b3d4aa68865aeb498b5c9cf (127.18.154.2:44527): tablet ff37fb2014f944bc90ef7fd8a6b0dbac (table test-table [id=d5560c32e41e4faf8acb0cc38d41ef86]) successfully deleted
Aug 07 18:17:47 dist-test-slave-gkw7 krb5kdc[14247](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126664, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.154.1@KRBTEST.COM
I20260807 18:17:47.364876 14289 catalog_manager.cc:4841] TS e19b02eadb8c49f7922095aca8e5a859 (127.18.154.3:37743): tablet ff37fb2014f944bc90ef7fd8a6b0dbac (table test-table [id=d5560c32e41e4faf8acb0cc38d41ef86]) successfully deleted
I20260807 18:17:47.371227 14985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:47.334069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57686 (local address 127.18.154.1:41381)
0807 18:17:47.334283 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:47.334289 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:47.342107 (+ 7818us) server_negotiation.cc:407] Connection header received
0807 18:17:47.342393 (+ 286us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:47.342400 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:47.342522 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:47.342609 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:17:47.344706 (+ 2097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:47.345977 (+ 1271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:47.347963 (+ 1986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:47.348631 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:47.348750 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:47.364715 (+ 15965us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:17:47.364733 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:17:47.364752 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:17:47.364801 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:17:47.368233 (+ 3432us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:17:47.369010 (+ 777us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:17:47.369017 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:17:47.369020 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:17:47.369160 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:17:47.369595 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:17:47.369603 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:17:47.369608 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:17:47.369988 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:17:47.370316 (+ 328us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:47.370644 (+ 328us) server_negotiation.cc:299] Negotiation successful
0807 18:17:47.370875 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260807 18:17:47.371976 14570 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:17:47.467983 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 14399
I20260807 18:17:47.504333 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 14587
Aug 07 18:17:47 dist-test-slave-gkw7 krb5kdc[14247](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126664, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.154.2@KRBTEST.COM
I20260807 18:17:47.523581 14979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:47.505276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45108 (local address 127.18.154.2:44527)
0807 18:17:47.505589 (+ 313us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:47.505602 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:47.505627 (+ 25us) server_negotiation.cc:407] Connection header received
0807 18:17:47.505770 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:47.505777 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:47.505876 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:47.506014 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:17:47.507487 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:47.508348 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:47.510645 (+ 2297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:47.511145 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:47.511355 (+ 210us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:47.517070 (+ 5715us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:17:47.517087 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:17:47.517105 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:17:47.517151 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:17:47.520478 (+ 3327us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:17:47.521232 (+ 754us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:17:47.521240 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:17:47.521242 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:17:47.521412 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:17:47.521880 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:17:47.521884 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:17:47.521886 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:17:47.522305 (+ 419us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:17:47.522616 (+ 311us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:47.522973 (+ 357us) server_negotiation.cc:299] Negotiation successful
0807 18:17:47.523235 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115}
I20260807 18:17:47.524333 14766 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:17:47.641851 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 14587
I20260807 18:17:47.667733 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 14779
Aug 07 18:17:47 dist-test-slave-gkw7 krb5kdc[14247](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126664, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.154.3@KRBTEST.COM
I20260807 18:17:47.684737 14983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:47.668552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43250 (local address 127.18.154.3:37743)
0807 18:17:47.668741 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:47.668745 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:47.668786 (+ 41us) server_negotiation.cc:407] Connection header received
0807 18:17:47.668983 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:47.668987 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:47.669054 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:47.669140 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:17:47.670749 (+ 1609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:47.671514 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:47.672932 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:47.673319 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:47.673409 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:47.678554 (+ 5145us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:17:47.678571 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:17:47.678588 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:17:47.678632 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:17:47.681909 (+ 3277us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:17:47.682764 (+ 855us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:17:47.682772 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:17:47.682775 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:17:47.682914 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:17:47.683266 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:17:47.683273 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:17:47.683276 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:17:47.683662 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:17:47.683876 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:47.684153 (+ 277us) server_negotiation.cc:299] Negotiation successful
0807 18:17:47.684322 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260807 18:17:47.685473 14955 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:17:47.776710 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 14779
I20260807 18:17:47.802474 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 14265
I20260807 18:17:47.803740 14353 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:17:47.917927 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 14265
2026-08-07T18:17:47Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (5132 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-07T18:17:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:17:47Z Disabled control of system clock
I20260807 18:17:48.022773 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126328808105-19048-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126328808105-19048-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126328808105-19048-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.154.62:36675
--webserver_interface=127.18.154.62
--webserver_port=0
--builtin_ntp_servers=127.18.154.20:34727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.154.62:36675
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 18:17:48.382135 15044 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:17:48.382467 15044 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:17:48.382637 15044 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:17:48.390110 15044 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:17:48.390224 15044 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:17:48.390264 15044 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:17:48.390379 15044 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:17:48.390473 15044 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:17:48.400049 15044 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.154.20:34727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126328808105-19048-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126328808105-19048-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.154.62:36675
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.154.62:36675
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.154.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126328808105-19048-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:17:48.401487 15044 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:17:48.403447 15044 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:17:48.416366 15053 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:17:48.416332 15057 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:17:48.416318 15055 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:17:48.417119 15044 server_base.cc:1034] running on GCE node
I20260807 18:17:48.418071 15044 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:17:48.419564 15044 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:17:48.420814 15044 hybrid_clock.cc:648] HybridClock initialized: now 1786126668420757 us; error 51 us; skew 500 ppm
I20260807 18:17:48.421187 15044 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 18:17:48.423622 15044 webserver.cc:460] Webserver started at http://127.18.154.62:37349/ using document root <none> and password file <none>
I20260807 18:17:48.424381 15044 fs_manager.cc:359] Metadata directory not provided
I20260807 18:17:48.424525 15044 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:17:48.424876 15044 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:17:48.427986 15044 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126328808105-19048-0/minicluster-data/master-0/data/instance:
uuid: "c45de265a0bf4830b61eddce85b2f220"
format_stamp: "Formatted at 2026-08-07 18:17:48 on dist-test-slave-gkw7"
server_key: "9202688971afa8433aadbfe8619ba058"
server_key_iv: "c70e12696e7c822eb81b6707cbec3d0a"
server_key_version: "encryptionkey@0"
I20260807 18:17:48.428869 15044 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126328808105-19048-0/minicluster-data/master-0/wal/instance:
uuid: "c45de265a0bf4830b61eddce85b2f220"
format_stamp: "Formatted at 2026-08-07 18:17:48 on dist-test-slave-gkw7"
server_key: "9202688971afa8433aadbfe8619ba058"
server_key_iv: "c70e12696e7c822eb81b6707cbec3d0a"
server_key_version: "encryptionkey@0"
I20260807 18:17:48.434396 15044 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.003s
I20260807 18:17:48.438082 15065 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:17:48.439776 15044 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 18:17:48.440052 15044 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126328808105-19048-0/minicluster-data/master-0/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126328808105-19048-0/minicluster-data/master-0/wal
uuid: "c45de265a0bf4830b61eddce85b2f220"
format_stamp: "Formatted at 2026-08-07 18:17:48 on dist-test-slave-gkw7"
server_key: "9202688971afa8433aadbfe8619ba058"
server_key_iv: "c70e12696e7c822eb81b6707cbec3d0a"
server_key_version: "encryptionkey@0"
I20260807 18:17:48.440348 15044 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126328808105-19048-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126328808105-19048-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126328808105-19048-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:17:48.459314 15044 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:17:48.460299 15044 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:17:48.460626 15044 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:17:48.480469 15044 rpc_server.cc:307] RPC server started. Bound to: 127.18.154.62:36675
I20260807 18:17:48.480481 15134 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.154.62:36675 every 8 connection(s)
I20260807 18:17:48.481984 15044 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
I20260807 18:17:48.485412 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 15044
I20260807 18:17:48.485628 19048 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126328808105-19048-0/minicluster-data/master-0/wal/instance
I20260807 18:17:48.486234 19048 external_mini_cluster.cc:1428] Setting key b82842a35b858269108795c24bb18a72
I20260807 18:17:48.486353 15135 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:17:48.499764 15135 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c45de265a0bf4830b61eddce85b2f220: Bootstrap starting.
I20260807 18:17:48.500926 15138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:48.488275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46494 (local address 127.18.154.62:36675)
0807 18:17:48.489368 (+ 1093us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:48.489379 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:48.489427 (+ 48us) server_negotiation.cc:407] Connection header received
0807 18:17:48.491640 (+ 2213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:48.491665 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:48.492182 (+ 517us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:48.492665 (+ 483us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:17:48.493322 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:48.494726 (+ 1404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:48.496121 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:48.496864 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:48.497110 (+ 246us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:48.498455 (+ 1345us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:17:48.498464 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:17:48.498479 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:17:48.499047 (+ 568us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:17:48.499128 (+ 81us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:17:48.499150 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:17:48.499300 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:48.499520 (+ 220us) server_negotiation.cc:299] Negotiation successful
0807 18:17:48.499860 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":604,"thread_start_us":375,"threads_started":1}
I20260807 18:17:48.503752 15135 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c45de265a0bf4830b61eddce85b2f220: Neither blocks nor log segments found. Creating new log.
I20260807 18:17:48.505019 15135 log.cc:824] T 00000000000000000000000000000000 P c45de265a0bf4830b61eddce85b2f220: Log is configured to *not* fsync() on all Append() calls
I20260807 18:17:48.507897 15135 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c45de265a0bf4830b61eddce85b2f220: No bootstrap required, opened a new log
I20260807 18:17:48.515955 15135 raft_consensus.cc:348] T 00000000000000000000000000000000 P c45de265a0bf4830b61eddce85b2f220 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c45de265a0bf4830b61eddce85b2f220" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 36675 } }
I20260807 18:17:48.516410 15135 raft_consensus.cc:374] T 00000000000000000000000000000000 P c45de265a0bf4830b61eddce85b2f220 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:17:48.516530 15135 raft_consensus.cc:729] T 00000000000000000000000000000000 P c45de265a0bf4830b61eddce85b2f220 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c45de265a0bf4830b61eddce85b2f220, State: Initialized, Role: FOLLOWER
I20260807 18:17:48.517280 15135 consensus_queue.cc:260] T 00000000000000000000000000000000 P c45de265a0bf4830b61eddce85b2f220 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c45de265a0bf4830b61eddce85b2f220" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 36675 } }
I20260807 18:17:48.517611 15135 raft_consensus.cc:388] T 00000000000000000000000000000000 P c45de265a0bf4830b61eddce85b2f220 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:17:48.517761 15135 raft_consensus.cc:482] T 00000000000000000000000000000000 P c45de265a0bf4830b61eddce85b2f220 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:17:48.517901 15135 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c45de265a0bf4830b61eddce85b2f220 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:17:48.520035 15135 raft_consensus.cc:504] T 00000000000000000000000000000000 P c45de265a0bf4830b61eddce85b2f220 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c45de265a0bf4830b61eddce85b2f220" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 36675 } }
I20260807 18:17:48.520509 15135 leader_election.cc:302] T 00000000000000000000000000000000 P c45de265a0bf4830b61eddce85b2f220 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c45de265a0bf4830b61eddce85b2f220; no voters:
I20260807 18:17:48.521209 15135 leader_election.cc:288] T 00000000000000000000000000000000 P c45de265a0bf4830b61eddce85b2f220 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:17:48.521409 15140 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c45de265a0bf4830b61eddce85b2f220 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:17:48.522678 15140 raft_consensus.cc:686] T 00000000000000000000000000000000 P c45de265a0bf4830b61eddce85b2f220 [term 1 LEADER]: Becoming Leader. State: Replica: c45de265a0bf4830b61eddce85b2f220, State: Running, Role: LEADER
I20260807 18:17:48.523293 15140 consensus_queue.cc:237] T 00000000000000000000000000000000 P c45de265a0bf4830b61eddce85b2f220 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c45de265a0bf4830b61eddce85b2f220" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 36675 } }
I20260807 18:17:48.523669 15135 sys_catalog.cc:564] T 00000000000000000000000000000000 P c45de265a0bf4830b61eddce85b2f220 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:17:48.526948 15142 sys_catalog.cc:455] T 00000000000000000000000000000000 P c45de265a0bf4830b61eddce85b2f220 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c45de265a0bf4830b61eddce85b2f220. Latest consensus state: current_term: 1 leader_uuid: "c45de265a0bf4830b61eddce85b2f220" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c45de265a0bf4830b61eddce85b2f220" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 36675 } } }
I20260807 18:17:48.527022 15141 sys_catalog.cc:455] T 00000000000000000000000000000000 P c45de265a0bf4830b61eddce85b2f220 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c45de265a0bf4830b61eddce85b2f220" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c45de265a0bf4830b61eddce85b2f220" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 36675 } } }
I20260807 18:17:48.527508 15142 sys_catalog.cc:458] T 00000000000000000000000000000000 P c45de265a0bf4830b61eddce85b2f220 [sys.catalog]: This master's current role is: LEADER
I20260807 18:17:48.527594 15141 sys_catalog.cc:458] T 00000000000000000000000000000000 P c45de265a0bf4830b61eddce85b2f220 [sys.catalog]: This master's current role is: LEADER
I20260807 18:17:48.529101 15150 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:17:48.534869 15150 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:17:48.544325 15150 catalog_manager.cc:1347] Generated new cluster ID: 2dfe01fbc3b14ed39da2638f80288238
I20260807 18:17:48.544530 15150 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:17:48.565681 15150 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:17:48.567072 15150 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:17:48.577428 15150 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c45de265a0bf4830b61eddce85b2f220: Generated new TSK 0
I20260807 18:17:48.578619 15150 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:17:48.591967 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126328808105-19048-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126328808105-19048-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126328808105-19048-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126328808105-19048-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.154.1:0
--local_ip_for_outbound_sockets=127.18.154.1
--webserver_interface=127.18.154.1
--webserver_port=0
--tserver_master_addrs=127.18.154.62:36675
--builtin_ntp_servers=127.18.154.20:34727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 18:17:48.947176 15170 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:17:48.947456 15170 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:17:48.947541 15170 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:17:48.954990 15170 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:17:48.955092 15170 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:17:48.955200 15170 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.154.1
I20260807 18:17:48.964337 15170 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.154.20:34727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126328808105-19048-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126328808105-19048-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.154.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126328808105-19048-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.154.1
--webserver_port=0
--tserver_master_addrs=127.18.154.62:36675
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.154.1
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126328808105-19048-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:17:48.965616 15170 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:17:48.967413 15170 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:17:48.981231 15178 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:17:48.981321 15180 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:17:48.981320 15177 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:17:48.982458 15170 server_base.cc:1034] running on GCE node
I20260807 18:17:48.983213 15170 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:17:48.984232 15170 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:17:48.985472 15170 hybrid_clock.cc:648] HybridClock initialized: now 1786126668985398 us; error 68 us; skew 500 ppm
I20260807 18:17:48.985826 15170 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 18:17:48.988166 15170 webserver.cc:460] Webserver started at http://127.18.154.1:37317/ using document root <none> and password file <none>
I20260807 18:17:48.988901 15170 fs_manager.cc:359] Metadata directory not provided
I20260807 18:17:48.989038 15170 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:17:48.989365 15170 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:17:48.992321 15170 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126328808105-19048-0/minicluster-data/ts-0/data/instance:
uuid: "9246b46749d644a29f19368f9ae89d09"
format_stamp: "Formatted at 2026-08-07 18:17:48 on dist-test-slave-gkw7"
server_key: "247f5fa838cd136da02cc9429732db26"
server_key_iv: "3fd82f02c77b9115e63dee5eb61cc4b7"
server_key_version: "encryptionkey@0"
I20260807 18:17:48.993134 15170 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126328808105-19048-0/minicluster-data/ts-0/wal/instance:
uuid: "9246b46749d644a29f19368f9ae89d09"
format_stamp: "Formatted at 2026-08-07 18:17:48 on dist-test-slave-gkw7"
server_key: "247f5fa838cd136da02cc9429732db26"
server_key_iv: "3fd82f02c77b9115e63dee5eb61cc4b7"
server_key_version: "encryptionkey@0"
I20260807 18:17:48.998471 15170 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.004s
I20260807 18:17:49.002087 15188 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:17:49.003592 15170 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 18:17:49.003734 15170 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126328808105-19048-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126328808105-19048-0/minicluster-data/ts-0/wal
uuid: "9246b46749d644a29f19368f9ae89d09"
format_stamp: "Formatted at 2026-08-07 18:17:48 on dist-test-slave-gkw7"
server_key: "247f5fa838cd136da02cc9429732db26"
server_key_iv: "3fd82f02c77b9115e63dee5eb61cc4b7"
server_key_version: "encryptionkey@0"
I20260807 18:17:49.003918 15170 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126328808105-19048-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126328808105-19048-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126328808105-19048-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:17:49.016343 15170 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:17:49.017140 15170 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:17:49.017409 15170 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:17:49.018442 15170 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:17:49.019897 15170 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:17:49.019986 15170 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:17:49.020077 15170 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:17:49.020133 15170 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:17:49.056089 15170 rpc_server.cc:307] RPC server started. Bound to: 127.18.154.1:40823
I20260807 18:17:49.056183 15343 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.154.1:40823 every 8 connection(s)
I20260807 18:17:49.057487 15170 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126328808105-19048-0/minicluster-data/ts-0/data/info.pb
I20260807 18:17:49.064826 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 15170
I20260807 18:17:49.065016 19048 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126328808105-19048-0/minicluster-data/ts-0/wal/instance
I20260807 18:17:49.065570 19048 external_mini_cluster.cc:1428] Setting key 0e55758212e739478a06e368bd18f10c
I20260807 18:17:49.068714 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126328808105-19048-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126328808105-19048-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126328808105-19048-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126328808105-19048-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.154.2:0
--local_ip_for_outbound_sockets=127.18.154.2
--webserver_interface=127.18.154.2
--webserver_port=0
--tserver_master_addrs=127.18.154.62:36675
--builtin_ntp_servers=127.18.154.20:34727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 18:17:49.072511 15347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:49.060681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.1:54705 (local address 127.18.154.62:36675)
0807 18:17:49.061177 (+ 496us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:49.061182 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:49.062509 (+ 1327us) server_negotiation.cc:407] Connection header received
0807 18:17:49.064468 (+ 1959us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:49.064473 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:49.064577 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:49.064707 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:17:49.066126 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:49.066971 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:49.069348 (+ 2377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:49.069888 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:49.070037 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:49.071566 (+ 1529us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:17:49.071573 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:17:49.071578 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:17:49.071610 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:17:49.071653 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:17:49.071665 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:17:49.071770 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:49.072028 (+ 258us) server_negotiation.cc:299] Negotiation successful
0807 18:17:49.072144 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":369,"thread_start_us":243,"threads_started":1}
I20260807 18:17:49.073683 15348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:49.061326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:36675 (local address 127.18.154.1:54705)
0807 18:17:49.062328 (+ 1002us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:49.062380 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0807 18:17:49.064177 (+ 1797us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:17:49.065015 (+ 838us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:17:49.065023 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:17:49.065165 (+ 142us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:17:49.065930 (+ 765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:49.065951 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:49.067138 (+ 1187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:49.067144 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:49.069191 (+ 2047us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:49.069205 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:49.070150 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:49.070162 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:49.070415 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:49.071218 (+ 803us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 18:17:49.071220 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:17:49.071264 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 18:17:49.071347 (+ 83us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:17:49.071769 (+ 422us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:17:49.071784 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:17:49.071796 (+ 12us) client_negotiation.cc:769] Sending connection context
0807 18:17:49.072003 (+ 207us) client_negotiation.cc:240] Negotiation successful
0807 18:17:49.072435 (+ 432us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":532,"thread_start_us":349,"threads_started":1}
I20260807 18:17:49.075378 15344 heartbeater.cc:347] Connected to a master server at 127.18.154.62:36675
I20260807 18:17:49.075732 15344 heartbeater.cc:464] Registering TS with master...
I20260807 18:17:49.076742 15344 heartbeater.cc:511] Master 127.18.154.62:36675 requested a full tablet report, sending...
I20260807 18:17:49.079409 15089 ts_manager.cc:194] Registered new tserver with Master: 9246b46749d644a29f19368f9ae89d09 (127.18.154.1:40823)
I20260807 18:17:49.081513 15089 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.154.1:54705
W20260807 18:17:49.418519 15349 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:17:49.418857 15349 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:17:49.419015 15349 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:17:49.426525 15349 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:17:49.426662 15349 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:17:49.426853 15349 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.154.2
I20260807 18:17:49.436093 15349 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.154.20:34727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126328808105-19048-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126328808105-19048-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.154.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126328808105-19048-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.154.2
--webserver_port=0
--tserver_master_addrs=127.18.154.62:36675
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.154.2
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126328808105-19048-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:17:49.437484 15349 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:17:49.439345 15349 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:17:49.453111 15354 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:17:49.453032 15358 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:17:49.453150 15355 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:17:49.453850 15349 server_base.cc:1034] running on GCE node
I20260807 18:17:49.454473 15349 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:17:49.455528 15349 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:17:49.456773 15349 hybrid_clock.cc:648] HybridClock initialized: now 1786126669456723 us; error 48 us; skew 500 ppm
I20260807 18:17:49.457105 15349 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 18:17:49.459450 15349 webserver.cc:460] Webserver started at http://127.18.154.2:35503/ using document root <none> and password file <none>
I20260807 18:17:49.460178 15349 fs_manager.cc:359] Metadata directory not provided
I20260807 18:17:49.460316 15349 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:17:49.460638 15349 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:17:49.463696 15349 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126328808105-19048-0/minicluster-data/ts-1/data/instance:
uuid: "45e69362007d48518105130cc5880d9c"
format_stamp: "Formatted at 2026-08-07 18:17:49 on dist-test-slave-gkw7"
server_key: "d8134fa63d56fde7f8408609c19938b4"
server_key_iv: "46b19aeb25fae3ed685be55ac9310f07"
server_key_version: "encryptionkey@0"
I20260807 18:17:49.464504 15349 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126328808105-19048-0/minicluster-data/ts-1/wal/instance:
uuid: "45e69362007d48518105130cc5880d9c"
format_stamp: "Formatted at 2026-08-07 18:17:49 on dist-test-slave-gkw7"
server_key: "d8134fa63d56fde7f8408609c19938b4"
server_key_iv: "46b19aeb25fae3ed685be55ac9310f07"
server_key_version: "encryptionkey@0"
I20260807 18:17:49.469640 15349 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260807 18:17:49.473307 15373 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:17:49.474658 15349 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260807 18:17:49.474848 15349 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126328808105-19048-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126328808105-19048-0/minicluster-data/ts-1/wal
uuid: "45e69362007d48518105130cc5880d9c"
format_stamp: "Formatted at 2026-08-07 18:17:49 on dist-test-slave-gkw7"
server_key: "d8134fa63d56fde7f8408609c19938b4"
server_key_iv: "46b19aeb25fae3ed685be55ac9310f07"
server_key_version: "encryptionkey@0"
I20260807 18:17:49.475068 15349 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126328808105-19048-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126328808105-19048-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126328808105-19048-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:17:49.498281 15349 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:17:49.499331 15349 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:17:49.499693 15349 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:17:49.500926 15349 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:17:49.502791 15349 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:17:49.502902 15349 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:17:49.503005 15349 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:17:49.503079 15349 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:17:49.539985 15349 rpc_server.cc:307] RPC server started. Bound to: 127.18.154.2:43559
I20260807 18:17:49.540093 15528 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.154.2:43559 every 8 connection(s)
I20260807 18:17:49.541348 15349 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126328808105-19048-0/minicluster-data/ts-1/data/info.pb
I20260807 18:17:49.543061 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 15349
I20260807 18:17:49.543242 19048 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126328808105-19048-0/minicluster-data/ts-1/wal/instance
I20260807 18:17:49.543735 19048 external_mini_cluster.cc:1428] Setting key f239658c177cd7cdd26aac23ebb3129e
I20260807 18:17:49.547194 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126328808105-19048-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126328808105-19048-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126328808105-19048-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126328808105-19048-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.154.3:0
--local_ip_for_outbound_sockets=127.18.154.3
--webserver_interface=127.18.154.3
--webserver_port=0
--tserver_master_addrs=127.18.154.62:36675
--builtin_ntp_servers=127.18.154.20:34727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 18:17:49.557483 15347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:49.544413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.2:48129 (local address 127.18.154.62:36675)
0807 18:17:49.544642 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:49.544649 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:49.546489 (+ 1840us) server_negotiation.cc:407] Connection header received
0807 18:17:49.549365 (+ 2876us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:49.549369 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:49.549439 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:49.549600 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:17:49.551087 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:49.551836 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:49.554371 (+ 2535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:49.554769 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:49.554876 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:49.556658 (+ 1782us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:17:49.556664 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:17:49.556668 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:17:49.556689 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:17:49.556725 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:17:49.556739 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:17:49.556820 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:49.557132 (+ 312us) server_negotiation.cc:299] Negotiation successful
0807 18:17:49.557239 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260807 18:17:49.559010 15531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:49.545065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:36675 (local address 127.18.154.2:48129)
0807 18:17:49.546316 (+ 1251us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:49.546367 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0807 18:17:49.549032 (+ 2665us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:17:49.549841 (+ 809us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:17:49.549854 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:17:49.549982 (+ 128us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:17:49.550893 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:49.550924 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:49.552008 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:49.552016 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:49.554202 (+ 2186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:49.554221 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:49.555007 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:49.555016 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:49.555359 (+ 343us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:49.556207 (+ 848us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 18:17:49.556213 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:17:49.556279 (+ 66us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 18:17:49.556393 (+ 114us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:17:49.556848 (+ 455us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:17:49.556861 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:17:49.556873 (+ 12us) client_negotiation.cc:769] Sending connection context
0807 18:17:49.557060 (+ 187us) client_negotiation.cc:240] Negotiation successful
0807 18:17:49.557512 (+ 452us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":630,"thread_start_us":341,"threads_started":1}
I20260807 18:17:49.560683 15529 heartbeater.cc:347] Connected to a master server at 127.18.154.62:36675
I20260807 18:17:49.561046 15529 heartbeater.cc:464] Registering TS with master...
I20260807 18:17:49.562187 15529 heartbeater.cc:511] Master 127.18.154.62:36675 requested a full tablet report, sending...
I20260807 18:17:49.563741 15089 ts_manager.cc:194] Registered new tserver with Master: 45e69362007d48518105130cc5880d9c (127.18.154.2:43559)
I20260807 18:17:49.564568 15089 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.154.2:48129
W20260807 18:17:49.906942 15532 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:17:49.907231 15532 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:17:49.907399 15532 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:17:49.914937 15532 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:17:49.915055 15532 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:17:49.915288 15532 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.154.3
I20260807 18:17:49.924750 15532 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.154.20:34727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126328808105-19048-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126328808105-19048-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.154.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126328808105-19048-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.154.3
--webserver_port=0
--tserver_master_addrs=127.18.154.62:36675
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.154.3
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126328808105-19048-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:17:49.926142 15532 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:17:49.928015 15532 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:17:49.941609 15541 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:17:49.941646 15538 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:17:49.941519 15539 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:17:49.942672 15532 server_base.cc:1034] running on GCE node
I20260807 18:17:49.943313 15532 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:17:49.944258 15532 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:17:49.945581 15532 hybrid_clock.cc:648] HybridClock initialized: now 1786126669945486 us; error 41 us; skew 500 ppm
I20260807 18:17:49.945993 15532 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 18:17:49.948725 15532 webserver.cc:460] Webserver started at http://127.18.154.3:38693/ using document root <none> and password file <none>
I20260807 18:17:49.949506 15532 fs_manager.cc:359] Metadata directory not provided
I20260807 18:17:49.949697 15532 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:17:49.950044 15532 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:17:49.953470 15532 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126328808105-19048-0/minicluster-data/ts-2/data/instance:
uuid: "9b817f145620410b86164c995aa326c9"
format_stamp: "Formatted at 2026-08-07 18:17:49 on dist-test-slave-gkw7"
server_key: "7af2b676835992ddcf07b46216524f9f"
server_key_iv: "b31cb3d8122980e679ee49b9faa4356f"
server_key_version: "encryptionkey@0"
I20260807 18:17:49.954490 15532 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126328808105-19048-0/minicluster-data/ts-2/wal/instance:
uuid: "9b817f145620410b86164c995aa326c9"
format_stamp: "Formatted at 2026-08-07 18:17:49 on dist-test-slave-gkw7"
server_key: "7af2b676835992ddcf07b46216524f9f"
server_key_iv: "b31cb3d8122980e679ee49b9faa4356f"
server_key_version: "encryptionkey@0"
I20260807 18:17:49.960600 15532 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.001s sys 0.005s
I20260807 18:17:49.964859 15548 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:17:49.966490 15532 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:17:49.966702 15532 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126328808105-19048-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126328808105-19048-0/minicluster-data/ts-2/wal
uuid: "9b817f145620410b86164c995aa326c9"
format_stamp: "Formatted at 2026-08-07 18:17:49 on dist-test-slave-gkw7"
server_key: "7af2b676835992ddcf07b46216524f9f"
server_key_iv: "b31cb3d8122980e679ee49b9faa4356f"
server_key_version: "encryptionkey@0"
I20260807 18:17:49.966974 15532 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126328808105-19048-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126328808105-19048-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126328808105-19048-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:17:50.021001 15532 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:17:50.022142 15532 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:17:50.022539 15532 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:17:50.023911 15532 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:17:50.025884 15532 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:17:50.026051 15532 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:17:50.026211 15532 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:17:50.026319 15532 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:17:50.065935 15532 rpc_server.cc:307] RPC server started. Bound to: 127.18.154.3:39549
I20260807 18:17:50.065984 15692 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.154.3:39549 every 8 connection(s)
I20260807 18:17:50.067317 15532 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126328808105-19048-0/minicluster-data/ts-2/data/info.pb
I20260807 18:17:50.074612 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 15532
I20260807 18:17:50.074811 19048 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126328808105-19048-0/minicluster-data/ts-2/wal/instance
I20260807 18:17:50.075253 19048 external_mini_cluster.cc:1428] Setting key 50d89c5ca973b8f7e52d9e483c7865b5
I20260807 18:17:50.081894 15696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:50.070010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.3:54293 (local address 127.18.154.62:36675)
0807 18:17:50.070496 (+ 486us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:50.070501 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:50.071886 (+ 1385us) server_negotiation.cc:407] Connection header received
0807 18:17:50.074175 (+ 2289us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:50.074183 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:50.074305 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:50.074454 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:17:50.075999 (+ 1545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:50.077113 (+ 1114us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:50.079091 (+ 1978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:50.079522 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:50.079619 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:50.081035 (+ 1416us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:17:50.081041 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:17:50.081046 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:17:50.081068 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:17:50.081113 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:17:50.081137 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:17:50.081214 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:50.081453 (+ 239us) server_negotiation.cc:299] Negotiation successful
0807 18:17:50.081589 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":375,"thread_start_us":241,"threads_started":1}
I20260807 18:17:50.082921 15698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:50.070644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:36675 (local address 127.18.154.3:54293)
0807 18:17:50.071681 (+ 1037us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:50.071723 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 18:17:50.073850 (+ 2127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:17:50.074795 (+ 945us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:17:50.074806 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:17:50.074953 (+ 147us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:17:50.075767 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:50.075798 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:50.077296 (+ 1498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:50.077300 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:50.078913 (+ 1613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:50.078951 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:50.079740 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:50.079746 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:50.079957 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:50.080718 (+ 761us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 18:17:50.080720 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:17:50.080762 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 18:17:50.080843 (+ 81us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:17:50.081232 (+ 389us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:17:50.081240 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:17:50.081246 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 18:17:50.081395 (+ 149us) client_negotiation.cc:240] Negotiation successful
0807 18:17:50.081810 (+ 415us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":526,"thread_start_us":333,"threads_started":1}
I20260807 18:17:50.084431 15693 heartbeater.cc:347] Connected to a master server at 127.18.154.62:36675
I20260807 18:17:50.084748 15693 heartbeater.cc:464] Registering TS with master...
I20260807 18:17:50.085126 15344 heartbeater.cc:503] Master 127.18.154.62:36675 was elected leader, sending a full tablet report...
I20260807 18:17:50.085790 15693 heartbeater.cc:511] Master 127.18.154.62:36675 requested a full tablet report, sending...
I20260807 18:17:50.087196 15089 ts_manager.cc:194] Registered new tserver with Master: 9b817f145620410b86164c995aa326c9 (127.18.154.3:39549)
I20260807 18:17:50.088011 15089 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.154.3:54293
I20260807 18:17:50.090809 19048 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 18:17:50.100790 15696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:50.095043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46508 (local address 127.18.154.62:36675)
0807 18:17:50.095295 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:50.095305 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:50.095667 (+ 362us) server_negotiation.cc:407] Connection header received
0807 18:17:50.095815 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:50.095819 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:50.095883 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:50.095966 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:17:50.096448 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:50.097136 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:50.098515 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:50.098917 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:50.099013 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:50.100099 (+ 1086us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:17:50.100104 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:17:50.100107 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:17:50.100124 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:17:50.100155 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:17:50.100167 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:17:50.100251 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:50.100386 (+ 135us) server_negotiation.cc:299] Negotiation successful
0807 18:17:50.100461 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260807 18:17:50.104702 15089 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46508:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 18:17:50.111888 15089 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:17:50.138099 15712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:50.125362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.1:40823 (local address 127.0.0.1:43300)
0807 18:17:50.126945 (+ 1583us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:50.126977 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0807 18:17:50.127117 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:17:50.129406 (+ 2289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:17:50.129412 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:17:50.129437 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:17:50.130007 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:50.130032 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:50.132100 (+ 2068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:50.132105 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:50.134690 (+ 2585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:50.134704 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:50.137089 (+ 2385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:50.137095 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:50.137314 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:50.137365 (+ 51us) client_negotiation.cc:769] Sending connection context
0807 18:17:50.137490 (+ 125us) client_negotiation.cc:240] Negotiation successful
0807 18:17:50.137736 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1357,"thread_start_us":1219,"threads_started":1}
I20260807 18:17:50.139292 15714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:50.126013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43300 (local address 127.18.154.1:40823)
0807 18:17:50.128579 (+ 2566us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:50.128594 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:50.128652 (+ 58us) server_negotiation.cc:407] Connection header received
0807 18:17:50.128787 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:50.128799 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:50.129147 (+ 348us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:50.129415 (+ 268us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:17:50.130262 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:50.131942 (+ 1680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:50.134933 (+ 2991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:50.136487 (+ 1554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:50.136634 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:50.136963 (+ 329us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:50.138685 (+ 1722us) server_negotiation.cc:299] Negotiation successful
0807 18:17:50.138902 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2337,"thread_start_us":2178,"threads_started":1}
I20260807 18:17:50.142477 15715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:50.125364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.3:39549 (local address 127.0.0.1:38882)
0807 18:17:50.130069 (+ 4705us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:50.130093 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 18:17:50.130208 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:17:50.131192 (+ 984us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:17:50.131197 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:17:50.131244 (+ 47us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:17:50.132800 (+ 1556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:50.132823 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:50.134851 (+ 2028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:50.134857 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:50.138022 (+ 3165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:50.138038 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:50.141488 (+ 3450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:50.141493 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:50.141706 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:50.141739 (+ 33us) client_negotiation.cc:769] Sending connection context
0807 18:17:50.141984 (+ 245us) client_negotiation.cc:240] Negotiation successful
0807 18:17:50.142128 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4517,"mutex_wait_us":912,"thread_start_us":3412,"threads_started":1}
I20260807 18:17:50.142588 15713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:50.125593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38882 (local address 127.18.154.3:39549)
0807 18:17:50.126625 (+ 1032us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:50.126636 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:50.130345 (+ 3709us) server_negotiation.cc:407] Connection header received
0807 18:17:50.130458 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:50.130464 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:50.130716 (+ 252us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:50.130906 (+ 190us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:17:50.133039 (+ 2133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:50.134679 (+ 1640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:50.138283 (+ 3604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:50.140633 (+ 2350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:50.141049 (+ 416us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:50.141367 (+ 318us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:50.142043 (+ 676us) server_negotiation.cc:299] Negotiation successful
0807 18:17:50.142211 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":779,"thread_start_us":636,"threads_started":1}
I20260807 18:17:50.144734 15250 tablet_service.cc:1467] Processing CreateTablet for tablet 0e0822a4c0af4b719a383f35c73ad5da (DEFAULT_TABLE table=test-table [id=51237dac9ed3488b9e19bbecaaa349c4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:17:50.146605 15250 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e0822a4c0af4b719a383f35c73ad5da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:17:50.149587 15608 tablet_service.cc:1467] Processing CreateTablet for tablet 0e0822a4c0af4b719a383f35c73ad5da (DEFAULT_TABLE table=test-table [id=51237dac9ed3488b9e19bbecaaa349c4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:17:50.150650 15712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:50.130203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.2:43559 (local address 127.0.0.1:46364)
0807 18:17:50.138401 (+ 8198us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:50.138419 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 18:17:50.138521 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:17:50.144750 (+ 6229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:17:50.144754 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:17:50.144791 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:17:50.145005 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:50.145017 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:50.146844 (+ 1827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:50.146849 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:50.148487 (+ 1638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:50.148500 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:50.149867 (+ 1367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:50.149873 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:50.150075 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:50.150108 (+ 33us) client_negotiation.cc:769] Sending connection context
0807 18:17:50.150211 (+ 103us) client_negotiation.cc:240] Negotiation successful
0807 18:17:50.150360 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8056,"thread_start_us":14844,"threads_started":1}
I20260807 18:17:50.151062 15717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:50.130498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46364 (local address 127.18.154.2:43559)
0807 18:17:50.144040 (+ 13542us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:50.144049 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:50.144083 (+ 34us) server_negotiation.cc:407] Connection header received
0807 18:17:50.144205 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:50.144213 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:50.144489 (+ 276us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:50.144802 (+ 313us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:17:50.145176 (+ 374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:50.146629 (+ 1453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:50.148646 (+ 2017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:50.149698 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:50.149863 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:50.150124 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:50.150260 (+ 136us) server_negotiation.cc:299] Negotiation successful
0807 18:17:50.150553 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13351,"thread_start_us":13946,"threads_started":1}
I20260807 18:17:50.151266 15608 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e0822a4c0af4b719a383f35c73ad5da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:17:50.157826 15443 tablet_service.cc:1467] Processing CreateTablet for tablet 0e0822a4c0af4b719a383f35c73ad5da (DEFAULT_TABLE table=test-table [id=51237dac9ed3488b9e19bbecaaa349c4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:17:50.159514 15443 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e0822a4c0af4b719a383f35c73ad5da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:17:50.165113 15718 tablet_bootstrap.cc:492] T 0e0822a4c0af4b719a383f35c73ad5da P 9246b46749d644a29f19368f9ae89d09: Bootstrap starting.
I20260807 18:17:50.167654 15719 tablet_bootstrap.cc:492] T 0e0822a4c0af4b719a383f35c73ad5da P 9b817f145620410b86164c995aa326c9: Bootstrap starting.
I20260807 18:17:50.168995 15718 tablet_bootstrap.cc:654] T 0e0822a4c0af4b719a383f35c73ad5da P 9246b46749d644a29f19368f9ae89d09: Neither blocks nor log segments found. Creating new log.
I20260807 18:17:50.170389 15718 log.cc:824] T 0e0822a4c0af4b719a383f35c73ad5da P 9246b46749d644a29f19368f9ae89d09: Log is configured to *not* fsync() on all Append() calls
I20260807 18:17:50.171552 15719 tablet_bootstrap.cc:654] T 0e0822a4c0af4b719a383f35c73ad5da P 9b817f145620410b86164c995aa326c9: Neither blocks nor log segments found. Creating new log.
I20260807 18:17:50.173048 15719 log.cc:824] T 0e0822a4c0af4b719a383f35c73ad5da P 9b817f145620410b86164c995aa326c9: Log is configured to *not* fsync() on all Append() calls
I20260807 18:17:50.177096 15719 tablet_bootstrap.cc:492] T 0e0822a4c0af4b719a383f35c73ad5da P 9b817f145620410b86164c995aa326c9: No bootstrap required, opened a new log
I20260807 18:17:50.177489 15719 ts_tablet_manager.cc:1397] T 0e0822a4c0af4b719a383f35c73ad5da P 9b817f145620410b86164c995aa326c9: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.009s
I20260807 18:17:50.177901 15723 tablet_bootstrap.cc:492] T 0e0822a4c0af4b719a383f35c73ad5da P 45e69362007d48518105130cc5880d9c: Bootstrap starting.
I20260807 18:17:50.179734 15718 tablet_bootstrap.cc:492] T 0e0822a4c0af4b719a383f35c73ad5da P 9246b46749d644a29f19368f9ae89d09: No bootstrap required, opened a new log
I20260807 18:17:50.180009 15718 ts_tablet_manager.cc:1397] T 0e0822a4c0af4b719a383f35c73ad5da P 9246b46749d644a29f19368f9ae89d09: Time spent bootstrapping tablet: real 0.015s user 0.004s sys 0.009s
I20260807 18:17:50.182956 15723 tablet_bootstrap.cc:654] T 0e0822a4c0af4b719a383f35c73ad5da P 45e69362007d48518105130cc5880d9c: Neither blocks nor log segments found. Creating new log.
I20260807 18:17:50.184376 15723 log.cc:824] T 0e0822a4c0af4b719a383f35c73ad5da P 45e69362007d48518105130cc5880d9c: Log is configured to *not* fsync() on all Append() calls
I20260807 18:17:50.187829 15723 tablet_bootstrap.cc:492] T 0e0822a4c0af4b719a383f35c73ad5da P 45e69362007d48518105130cc5880d9c: No bootstrap required, opened a new log
I20260807 18:17:50.187688 15718 raft_consensus.cc:348] T 0e0822a4c0af4b719a383f35c73ad5da P 9246b46749d644a29f19368f9ae89d09 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b817f145620410b86164c995aa326c9" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 39549 } } peers { permanent_uuid: "9246b46749d644a29f19368f9ae89d09" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 40823 } } peers { permanent_uuid: "45e69362007d48518105130cc5880d9c" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 43559 } }
I20260807 18:17:50.188141 15723 ts_tablet_manager.cc:1397] T 0e0822a4c0af4b719a383f35c73ad5da P 45e69362007d48518105130cc5880d9c: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.007s
I20260807 18:17:50.188177 15718 raft_consensus.cc:374] T 0e0822a4c0af4b719a383f35c73ad5da P 9246b46749d644a29f19368f9ae89d09 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:17:50.188275 15718 raft_consensus.cc:729] T 0e0822a4c0af4b719a383f35c73ad5da P 9246b46749d644a29f19368f9ae89d09 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9246b46749d644a29f19368f9ae89d09, State: Initialized, Role: FOLLOWER
I20260807 18:17:50.189047 15718 consensus_queue.cc:260] T 0e0822a4c0af4b719a383f35c73ad5da P 9246b46749d644a29f19368f9ae89d09 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b817f145620410b86164c995aa326c9" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 39549 } } peers { permanent_uuid: "9246b46749d644a29f19368f9ae89d09" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 40823 } } peers { permanent_uuid: "45e69362007d48518105130cc5880d9c" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 43559 } }
I20260807 18:17:50.189407 15719 raft_consensus.cc:348] T 0e0822a4c0af4b719a383f35c73ad5da P 9b817f145620410b86164c995aa326c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b817f145620410b86164c995aa326c9" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 39549 } } peers { permanent_uuid: "9246b46749d644a29f19368f9ae89d09" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 40823 } } peers { permanent_uuid: "45e69362007d48518105130cc5880d9c" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 43559 } }
I20260807 18:17:50.189960 15719 raft_consensus.cc:374] T 0e0822a4c0af4b719a383f35c73ad5da P 9b817f145620410b86164c995aa326c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:17:50.190739 15718 ts_tablet_manager.cc:1428] T 0e0822a4c0af4b719a383f35c73ad5da P 9246b46749d644a29f19368f9ae89d09: Time spent starting tablet: real 0.011s user 0.009s sys 0.001s
I20260807 18:17:50.190078 15719 raft_consensus.cc:729] T 0e0822a4c0af4b719a383f35c73ad5da P 9b817f145620410b86164c995aa326c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b817f145620410b86164c995aa326c9, State: Initialized, Role: FOLLOWER
I20260807 18:17:50.191557 15719 consensus_queue.cc:260] T 0e0822a4c0af4b719a383f35c73ad5da P 9b817f145620410b86164c995aa326c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b817f145620410b86164c995aa326c9" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 39549 } } peers { permanent_uuid: "9246b46749d644a29f19368f9ae89d09" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 40823 } } peers { permanent_uuid: "45e69362007d48518105130cc5880d9c" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 43559 } }
I20260807 18:17:50.192827 15693 heartbeater.cc:503] Master 127.18.154.62:36675 was elected leader, sending a full tablet report...
I20260807 18:17:50.193400 15719 ts_tablet_manager.cc:1428] T 0e0822a4c0af4b719a383f35c73ad5da P 9b817f145620410b86164c995aa326c9: Time spent starting tablet: real 0.016s user 0.011s sys 0.003s
I20260807 18:17:50.199296 15723 raft_consensus.cc:348] T 0e0822a4c0af4b719a383f35c73ad5da P 45e69362007d48518105130cc5880d9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b817f145620410b86164c995aa326c9" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 39549 } } peers { permanent_uuid: "9246b46749d644a29f19368f9ae89d09" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 40823 } } peers { permanent_uuid: "45e69362007d48518105130cc5880d9c" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 43559 } }
I20260807 18:17:50.199630 15723 raft_consensus.cc:374] T 0e0822a4c0af4b719a383f35c73ad5da P 45e69362007d48518105130cc5880d9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:17:50.199761 15723 raft_consensus.cc:729] T 0e0822a4c0af4b719a383f35c73ad5da P 45e69362007d48518105130cc5880d9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45e69362007d48518105130cc5880d9c, State: Initialized, Role: FOLLOWER
I20260807 18:17:50.200328 15723 consensus_queue.cc:260] T 0e0822a4c0af4b719a383f35c73ad5da P 45e69362007d48518105130cc5880d9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b817f145620410b86164c995aa326c9" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 39549 } } peers { permanent_uuid: "9246b46749d644a29f19368f9ae89d09" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 40823 } } peers { permanent_uuid: "45e69362007d48518105130cc5880d9c" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 43559 } }
I20260807 18:17:50.201489 15529 heartbeater.cc:503] Master 127.18.154.62:36675 was elected leader, sending a full tablet report...
I20260807 18:17:50.201903 15723 ts_tablet_manager.cc:1428] T 0e0822a4c0af4b719a383f35c73ad5da P 45e69362007d48518105130cc5880d9c: Time spent starting tablet: real 0.014s user 0.006s sys 0.009s
I20260807 18:17:50.281481 15725 raft_consensus.cc:482] T 0e0822a4c0af4b719a383f35c73ad5da P 9246b46749d644a29f19368f9ae89d09 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:17:50.281869 15725 raft_consensus.cc:504] T 0e0822a4c0af4b719a383f35c73ad5da P 9246b46749d644a29f19368f9ae89d09 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b817f145620410b86164c995aa326c9" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 39549 } } peers { permanent_uuid: "9246b46749d644a29f19368f9ae89d09" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 40823 } } peers { permanent_uuid: "45e69362007d48518105130cc5880d9c" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 43559 } }
I20260807 18:17:50.283619 15725 leader_election.cc:288] T 0e0822a4c0af4b719a383f35c73ad5da P 9246b46749d644a29f19368f9ae89d09 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9b817f145620410b86164c995aa326c9 (127.18.154.3:39549), 45e69362007d48518105130cc5880d9c (127.18.154.2:43559)
I20260807 18:17:50.290887 15728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:50.283934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.3:39549 (local address 127.18.154.1:44959)
0807 18:17:50.284624 (+ 690us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:50.284646 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 18:17:50.284806 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:17:50.285230 (+ 424us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:17:50.285235 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:17:50.285266 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:17:50.285749 (+ 483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:50.285763 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:50.287053 (+ 1290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:50.287058 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:50.288730 (+ 1672us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:50.288743 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:50.290157 (+ 1414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:50.290164 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:50.290330 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:50.290378 (+ 48us) client_negotiation.cc:769] Sending connection context
0807 18:17:50.290469 (+ 91us) client_negotiation.cc:240] Negotiation successful
0807 18:17:50.290613 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":534,"thread_start_us":367,"threads_started":1}
I20260807 18:17:50.290956 15713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:50.284031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.1:44959 (local address 127.18.154.3:39549)
0807 18:17:50.284224 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:50.284230 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:50.284740 (+ 510us) server_negotiation.cc:407] Connection header received
0807 18:17:50.284976 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:50.284982 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:50.285071 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:50.285177 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:17:50.285909 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:50.286925 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:50.288960 (+ 2035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:50.289976 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:50.290100 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:50.290152 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:50.290520 (+ 368us) server_negotiation.cc:299] Negotiation successful
0807 18:17:50.290687 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 18:17:50.291343 15729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:50.284757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.2:43559 (local address 127.18.154.1:34753)
0807 18:17:50.285440 (+ 683us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:50.285456 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 18:17:50.285629 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:17:50.286009 (+ 380us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:17:50.286013 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:17:50.286043 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:17:50.286405 (+ 362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:50.286417 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:50.287593 (+ 1176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:50.287598 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:50.289182 (+ 1584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:17:50.289197 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:17:50.290544 (+ 1347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:17:50.290550 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:17:50.290734 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:50.290767 (+ 33us) client_negotiation.cc:769] Sending connection context
0807 18:17:50.290861 (+ 94us) client_negotiation.cc:240] Negotiation successful
0807 18:17:50.290994 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":573,"thread_start_us":368,"threads_started":1}
I20260807 18:17:50.291353 15717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:50.284923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.154.1:34753 (local address 127.18.154.2:43559)
0807 18:17:50.285119 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:50.285127 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:50.285573 (+ 446us) server_negotiation.cc:407] Connection header received
0807 18:17:50.285770 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:50.285775 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:50.285871 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:50.285955 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:17:50.286560 (+ 605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:50.287414 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:50.289380 (+ 1966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:50.290311 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:50.290477 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:50.290540 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:50.290918 (+ 378us) server_negotiation.cc:299] Negotiation successful
0807 18:17:50.291060 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 18:17:50.291958 15631 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e0822a4c0af4b719a383f35c73ad5da" candidate_uuid: "9246b46749d644a29f19368f9ae89d09" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b817f145620410b86164c995aa326c9" is_pre_election: true
I20260807 18:17:50.292285 15470 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e0822a4c0af4b719a383f35c73ad5da" candidate_uuid: "9246b46749d644a29f19368f9ae89d09" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45e69362007d48518105130cc5880d9c" is_pre_election: true
I20260807 18:17:50.292484 15631 raft_consensus.cc:2454] T 0e0822a4c0af4b719a383f35c73ad5da P 9b817f145620410b86164c995aa326c9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9246b46749d644a29f19368f9ae89d09 in term 0.
I20260807 18:17:50.292850 15470 raft_consensus.cc:2454] T 0e0822a4c0af4b719a383f35c73ad5da P 45e69362007d48518105130cc5880d9c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9246b46749d644a29f19368f9ae89d09 in term 0.
I20260807 18:17:50.293242 15190 leader_election.cc:302] T 0e0822a4c0af4b719a383f35c73ad5da P 9246b46749d644a29f19368f9ae89d09 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9246b46749d644a29f19368f9ae89d09, 9b817f145620410b86164c995aa326c9; no voters:
W20260807 18:17:50.293521 15530 tablet.cc:2369] T 0e0822a4c0af4b719a383f35c73ad5da P 45e69362007d48518105130cc5880d9c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:17:50.293658 15725 raft_consensus.cc:2781] T 0e0822a4c0af4b719a383f35c73ad5da P 9246b46749d644a29f19368f9ae89d09 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:17:50.293817 15725 raft_consensus.cc:482] T 0e0822a4c0af4b719a383f35c73ad5da P 9246b46749d644a29f19368f9ae89d09 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:17:50.293931 15725 raft_consensus.cc:3037] T 0e0822a4c0af4b719a383f35c73ad5da P 9246b46749d644a29f19368f9ae89d09 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:17:50.297148 15725 raft_consensus.cc:504] T 0e0822a4c0af4b719a383f35c73ad5da P 9246b46749d644a29f19368f9ae89d09 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b817f145620410b86164c995aa326c9" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 39549 } } peers { permanent_uuid: "9246b46749d644a29f19368f9ae89d09" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 40823 } } peers { permanent_uuid: "45e69362007d48518105130cc5880d9c" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 43559 } }
I20260807 18:17:50.297958 15725 leader_election.cc:288] T 0e0822a4c0af4b719a383f35c73ad5da P 9246b46749d644a29f19368f9ae89d09 [CANDIDATE]: Term 1 election: Requested vote from peers 9b817f145620410b86164c995aa326c9 (127.18.154.3:39549), 45e69362007d48518105130cc5880d9c (127.18.154.2:43559)
I20260807 18:17:50.298482 15470 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e0822a4c0af4b719a383f35c73ad5da" candidate_uuid: "9246b46749d644a29f19368f9ae89d09" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45e69362007d48518105130cc5880d9c"
I20260807 18:17:50.298539 15631 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e0822a4c0af4b719a383f35c73ad5da" candidate_uuid: "9246b46749d644a29f19368f9ae89d09" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b817f145620410b86164c995aa326c9"
I20260807 18:17:50.298732 15470 raft_consensus.cc:3037] T 0e0822a4c0af4b719a383f35c73ad5da P 45e69362007d48518105130cc5880d9c [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:17:50.298775 15631 raft_consensus.cc:3037] T 0e0822a4c0af4b719a383f35c73ad5da P 9b817f145620410b86164c995aa326c9 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:17:50.301098 15631 raft_consensus.cc:2454] T 0e0822a4c0af4b719a383f35c73ad5da P 9b817f145620410b86164c995aa326c9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9246b46749d644a29f19368f9ae89d09 in term 1.
I20260807 18:17:50.301110 15470 raft_consensus.cc:2454] T 0e0822a4c0af4b719a383f35c73ad5da P 45e69362007d48518105130cc5880d9c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9246b46749d644a29f19368f9ae89d09 in term 1.
I20260807 18:17:50.301493 15190 leader_election.cc:302] T 0e0822a4c0af4b719a383f35c73ad5da P 9246b46749d644a29f19368f9ae89d09 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45e69362007d48518105130cc5880d9c, 9246b46749d644a29f19368f9ae89d09; no voters:
I20260807 18:17:50.301842 15725 raft_consensus.cc:2781] T 0e0822a4c0af4b719a383f35c73ad5da P 9246b46749d644a29f19368f9ae89d09 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:17:50.302332 15725 raft_consensus.cc:686] T 0e0822a4c0af4b719a383f35c73ad5da P 9246b46749d644a29f19368f9ae89d09 [term 1 LEADER]: Becoming Leader. State: Replica: 9246b46749d644a29f19368f9ae89d09, State: Running, Role: LEADER
I20260807 18:17:50.302832 15725 consensus_queue.cc:237] T 0e0822a4c0af4b719a383f35c73ad5da P 9246b46749d644a29f19368f9ae89d09 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b817f145620410b86164c995aa326c9" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 39549 } } peers { permanent_uuid: "9246b46749d644a29f19368f9ae89d09" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 40823 } } peers { permanent_uuid: "45e69362007d48518105130cc5880d9c" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 43559 } }
I20260807 18:17:50.307839 15089 catalog_manager.cc:5495] T 0e0822a4c0af4b719a383f35c73ad5da P 9246b46749d644a29f19368f9ae89d09 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9246b46749d644a29f19368f9ae89d09 (127.18.154.1). New cstate: current_term: 1 leader_uuid: "9246b46749d644a29f19368f9ae89d09" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b817f145620410b86164c995aa326c9" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 39549 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9246b46749d644a29f19368f9ae89d09" member_type: VOTER last_known_addr { host: "127.18.154.1" port: 40823 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "45e69362007d48518105130cc5880d9c" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 43559 } health_report { overall_health: UNKNOWN } } }
W20260807 18:17:50.310343 15346 tablet.cc:2369] T 0e0822a4c0af4b719a383f35c73ad5da P 9246b46749d644a29f19368f9ae89d09: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 18:17:50.319125 15694 tablet.cc:2369] T 0e0822a4c0af4b719a383f35c73ad5da P 9b817f145620410b86164c995aa326c9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:17:50.329947 15714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:50.323750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43316 (local address 127.18.154.1:40823)
0807 18:17:50.324009 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:50.324016 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:50.324144 (+ 128us) server_negotiation.cc:407] Connection header received
0807 18:17:50.324444 (+ 300us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:50.324452 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:50.324602 (+ 150us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:50.324765 (+ 163us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:17:50.325207 (+ 442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:50.326043 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:50.327606 (+ 1563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:50.328116 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:50.328256 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:50.328769 (+ 513us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:17:50.329292 (+ 523us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:17:50.329461 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:50.329599 (+ 138us) server_negotiation.cc:299] Negotiation successful
0807 18:17:50.329685 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100}
I20260807 18:17:50.339522 15631 raft_consensus.cc:1261] T 0e0822a4c0af4b719a383f35c73ad5da P 9b817f145620410b86164c995aa326c9 [term 1 FOLLOWER]: Refusing update from remote peer 9246b46749d644a29f19368f9ae89d09: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:17:50.339524 15470 raft_consensus.cc:1261] T 0e0822a4c0af4b719a383f35c73ad5da P 45e69362007d48518105130cc5880d9c [term 1 FOLLOWER]: Refusing update from remote peer 9246b46749d644a29f19368f9ae89d09: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:17:50.340590 15725 consensus_queue.cc:1035] T 0e0822a4c0af4b719a383f35c73ad5da P 9246b46749d644a29f19368f9ae89d09 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9b817f145620410b86164c995aa326c9" member_type: VOTER last_known_addr { host: "127.18.154.3" port: 39549 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:17:50.340956 15730 consensus_queue.cc:1035] T 0e0822a4c0af4b719a383f35c73ad5da P 9246b46749d644a29f19368f9ae89d09 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45e69362007d48518105130cc5880d9c" member_type: VOTER last_known_addr { host: "127.18.154.2" port: 43559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:17:50.354326 15734 mvcc.cc:205] Tried to move back new op lower bound from 7315974841697492992 to 7315974841564028928. Current Snapshot: MvccSnapshot[applied={T|T < 7315974841697492992}]
I20260807 18:17:50.370803 15088 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46508:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:17:50.371186 15088 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46508:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:17:50.375803 15088 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c45de265a0bf4830b61eddce85b2f220: Sending DeleteTablet for 3 replicas of tablet 0e0822a4c0af4b719a383f35c73ad5da
I20260807 18:17:50.377147 15250 tablet_service.cc:1514] Processing DeleteTablet for tablet 0e0822a4c0af4b719a383f35c73ad5da with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:17:50 UTC) from {username='slave'} at 127.0.0.1:43300
I20260807 18:17:50.377372 15443 tablet_service.cc:1514] Processing DeleteTablet for tablet 0e0822a4c0af4b719a383f35c73ad5da with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:17:50 UTC) from {username='slave'} at 127.0.0.1:46364
I20260807 18:17:50.377637 15608 tablet_service.cc:1514] Processing DeleteTablet for tablet 0e0822a4c0af4b719a383f35c73ad5da with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:17:50 UTC) from {username='slave'} at 127.0.0.1:38882
I20260807 18:17:50.379675 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 15170
I20260807 18:17:50.390460 15751 tablet_replica.cc:330] T 0e0822a4c0af4b719a383f35c73ad5da P 45e69362007d48518105130cc5880d9c: stopping tablet replica
I20260807 18:17:50.390986 15751 raft_consensus.cc:2229] T 0e0822a4c0af4b719a383f35c73ad5da P 45e69362007d48518105130cc5880d9c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:17:50.391578 15751 raft_consensus.cc:2258] T 0e0822a4c0af4b719a383f35c73ad5da P 45e69362007d48518105130cc5880d9c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:17:50.395747 15751 ts_tablet_manager.cc:1905] T 0e0822a4c0af4b719a383f35c73ad5da P 45e69362007d48518105130cc5880d9c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:17:50.403839 15750 tablet_replica.cc:330] T 0e0822a4c0af4b719a383f35c73ad5da P 9246b46749d644a29f19368f9ae89d09: stopping tablet replica
I20260807 18:17:50.404561 15750 raft_consensus.cc:2229] T 0e0822a4c0af4b719a383f35c73ad5da P 9246b46749d644a29f19368f9ae89d09 [term 1 LEADER]: Raft consensus shutting down.
I20260807 18:17:50.405001 15751 ts_tablet_manager.cc:1918] T 0e0822a4c0af4b719a383f35c73ad5da P 45e69362007d48518105130cc5880d9c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:17:50.405222 15751 log.cc:1196] T 0e0822a4c0af4b719a383f35c73ad5da P 45e69362007d48518105130cc5880d9c: Deleting WAL directory at /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126328808105-19048-0/minicluster-data/ts-1/wal/wals/0e0822a4c0af4b719a383f35c73ad5da
I20260807 18:17:50.405658 15750 raft_consensus.cc:2258] T 0e0822a4c0af4b719a383f35c73ad5da P 9246b46749d644a29f19368f9ae89d09 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:17:50.405900 15751 ts_tablet_manager.cc:1939] T 0e0822a4c0af4b719a383f35c73ad5da P 45e69362007d48518105130cc5880d9c: Deleting consensus metadata
I20260807 18:17:50.406745 15753 tablet_replica.cc:330] T 0e0822a4c0af4b719a383f35c73ad5da P 9b817f145620410b86164c995aa326c9: stopping tablet replica
I20260807 18:17:50.407372 15753 raft_consensus.cc:2229] T 0e0822a4c0af4b719a383f35c73ad5da P 9b817f145620410b86164c995aa326c9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:17:50.407990 15068 catalog_manager.cc:4841] TS 45e69362007d48518105130cc5880d9c (127.18.154.2:43559): tablet 0e0822a4c0af4b719a383f35c73ad5da (table test-table [id=51237dac9ed3488b9e19bbecaaa349c4]) successfully deleted
I20260807 18:17:50.409370 15753 raft_consensus.cc:2258] T 0e0822a4c0af4b719a383f35c73ad5da P 9b817f145620410b86164c995aa326c9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:17:50.409895 15750 ts_tablet_manager.cc:1905] T 0e0822a4c0af4b719a383f35c73ad5da P 9246b46749d644a29f19368f9ae89d09: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:17:50.411306 15714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:50.381846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43320 (local address 127.18.154.1:40823)
0807 18:17:50.382122 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:50.382134 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:50.395984 (+ 13850us) server_negotiation.cc:407] Connection header received
0807 18:17:50.396431 (+ 447us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:50.396443 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:50.396641 (+ 198us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:50.396761 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:17:50.397874 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:50.399795 (+ 1921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:50.406614 (+ 6819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:50.407519 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:50.407666 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:50.409408 (+ 1742us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:17:50.409420 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:17:50.409438 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:17:50.410256 (+ 818us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:17:50.410318 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:17:50.410343 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:17:50.410440 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:50.410795 (+ 355us) server_negotiation.cc:299] Negotiation successful
0807 18:17:50.410903 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260807 18:17:50.412397 15753 ts_tablet_manager.cc:1905] T 0e0822a4c0af4b719a383f35c73ad5da P 9b817f145620410b86164c995aa326c9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:17:50.412736 15338 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:17:50.421646 15753 ts_tablet_manager.cc:1918] T 0e0822a4c0af4b719a383f35c73ad5da P 9b817f145620410b86164c995aa326c9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:17:50.421906 15753 log.cc:1196] T 0e0822a4c0af4b719a383f35c73ad5da P 9b817f145620410b86164c995aa326c9: Deleting WAL directory at /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786126328808105-19048-0/minicluster-data/ts-2/wal/wals/0e0822a4c0af4b719a383f35c73ad5da
I20260807 18:17:50.422595 15753 ts_tablet_manager.cc:1939] T 0e0822a4c0af4b719a383f35c73ad5da P 9b817f145620410b86164c995aa326c9: Deleting consensus metadata
I20260807 18:17:50.425076 15068 catalog_manager.cc:4841] TS 9b817f145620410b86164c995aa326c9 (127.18.154.3:39549): tablet 0e0822a4c0af4b719a383f35c73ad5da (table test-table [id=51237dac9ed3488b9e19bbecaaa349c4]) successfully deleted
I20260807 18:17:50.518571 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 15170
I20260807 18:17:50.544263 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 15349
W20260807 18:17:50.544183 15070 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.154.1:40823 (error 108)
I20260807 18:17:50.546427 15712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:50.545873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.1:40823 (local address 127.0.0.1:43330)
0807 18:17:50.546085 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:50.546201 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.1:40823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260807 18:17:50.546725 15070 catalog_manager.cc:4584] TS 9246b46749d644a29f19368f9ae89d09 (127.18.154.1:40823): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0e0822a4c0af4b719a383f35c73ad5da: Network error: Client connection negotiation failed: client connection to 127.18.154.1:40823: connect: Connection refused (error 111)
I20260807 18:17:50.553748 15717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:50.545457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46372 (local address 127.18.154.2:43559)
0807 18:17:50.545650 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:50.545659 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:50.546190 (+ 531us) server_negotiation.cc:407] Connection header received
0807 18:17:50.546377 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:50.546384 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:50.546484 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:50.546602 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:17:50.547293 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:50.548492 (+ 1199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:50.550019 (+ 1527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:50.550709 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:50.550834 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:50.551978 (+ 1144us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:17:50.551991 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:17:50.552011 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:17:50.552842 (+ 831us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:17:50.552906 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:17:50.552933 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:17:50.553062 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:50.553166 (+ 104us) server_negotiation.cc:299] Negotiation successful
0807 18:17:50.553285 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 18:17:50.554490 15524 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:17:50.600584 15712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:50.600036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.1:40823 (local address 127.0.0.1:43338)
0807 18:17:50.600302 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:50.600417 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.1:40823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106}
I20260807 18:17:50.601857 15712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:50.601412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.1:40823 (local address 127.0.0.1:43352)
0807 18:17:50.601635 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:50.601707 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.1:40823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260807 18:17:50.646813 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 15349
I20260807 18:17:50.662760 15712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:50.662273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.1:40823 (local address 127.0.0.1:43360)
0807 18:17:50.662508 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:50.662603 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.1:40823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260807 18:17:50.663904 15712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:50.663530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.1:40823 (local address 127.0.0.1:43366)
0807 18:17:50.663689 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:50.663775 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.1:40823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260807 18:17:50.670987 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 15532
I20260807 18:17:50.678117 15713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:50.671835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38886 (local address 127.18.154.3:39549)
0807 18:17:50.672010 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:50.672015 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:50.672055 (+ 40us) server_negotiation.cc:407] Connection header received
0807 18:17:50.672120 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:50.672124 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:50.672210 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:50.672289 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:17:50.672665 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:50.673621 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:50.674886 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:50.675289 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:50.675385 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:50.676558 (+ 1173us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:17:50.676568 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:17:50.676587 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:17:50.677238 (+ 651us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:17:50.677288 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:17:50.677308 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:17:50.677395 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:50.677766 (+ 371us) server_negotiation.cc:299] Negotiation successful
0807 18:17:50.677868 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 18:17:50.678843 15688 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:17:50.744472 15712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:50.743961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.1:40823 (local address 127.0.0.1:43374)
0807 18:17:50.744215 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:50.744320 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.1:40823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103}
I20260807 18:17:50.745702 15712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:50.745175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.1:40823 (local address 127.0.0.1:43382)
0807 18:17:50.745411 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0807 18:17:50.745490 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.1:40823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125}
I20260807 18:17:50.769192 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 15532
I20260807 18:17:50.794641 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 15044
I20260807 18:17:50.795881 15129 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:17:50.907295 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 15044
2026-08-07T18:17:50Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (2988 ms)
[----------] 6 tests from EncryptionPolicyTest (24280 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786126328808105-19048-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:17:51 dist-test-slave-gkw7 krb5kdc[15766](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:17:51 dist-test-slave-gkw7 krb5kdc[15766](info): set up 2 sockets
Aug 07 18:17:51 dist-test-slave-gkw7 krb5kdc[15766](info): commencing operation
krb5kdc: starting...
W20260807 18:17:52.465655 19048 mini_kdc.cc:121] Time spent starting KDC: real 1.504s user 0.006s sys 0.042s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786126328808105-19048-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786126328808105-19048-0/krb5kdc/test-user.keytab.
Aug 07 18:17:52 dist-test-slave-gkw7 krb5kdc[15766](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126672, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
W20260807 18:17:52.873099 19048 mini_kdc.cc:317] Time spent kinit for test-admin: real 0.119s user 0.001s sys 0.047s
2026-08-07T18:17:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:17:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786126328808105-19048-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786126328808105-19048-0/krb5kdc/kudu_127.0.0.1.keytab.
W20260807 18:17:53.098408 19048 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.0.0.1: real 0.133s user 0.006s sys 0.045s
I20260807 18:17:53.101897 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786126328808105-19048-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786126328808105-19048-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786126328808105-19048-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37807
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786126328808105-19048-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37807
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786126328808105-19048-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786126328808105-19048-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786126328808105-19048-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:17:53.752864 15781 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:17:53.753242 15781 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:17:53.753425 15781 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:17:53.761278 15781 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:17:53.761400 15781 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:17:53.761478 15781 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:17:53.761598 15781 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:17:53.761668 15781 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:17:53.771651 15781 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786126328808105-19048-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786126328808105-19048-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37807
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786126328808105-19048-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37807
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786126328808105-19048-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:17:53.773025 15781 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:17:53.774505 15781 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:17:53.787067 15793 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:17:53.787153 15790 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:17:53.787412 15788 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:17:53.787881 15781 server_base.cc:1034] running on GCE node
I20260807 18:17:53.788800 15781 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:17:53.790371 15781 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:17:53.791616 15781 hybrid_clock.cc:648] HybridClock initialized: now 1786126673791577 us; error 54 us; skew 500 ppm
I20260807 18:17:53.791990 15781 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:17:53 dist-test-slave-gkw7 krb5kdc[15766](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126673, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:17:53.800696 15781 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 18:17:53.802409 15781 webserver.cc:460] Webserver started at http://127.0.0.1:33099/ using document root <none> and password file <none>
I20260807 18:17:53.803155 15781 fs_manager.cc:359] Metadata directory not provided
I20260807 18:17:53.803313 15781 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:17:53.803712 15781 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:17:53.806968 15781 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786126328808105-19048-0/minicluster-data/master-0/data/instance:
uuid: "2cd9df9d6aef47979a81b20edd4ad4c9"
format_stamp: "Formatted at 2026-08-07 18:17:53 on dist-test-slave-gkw7"
server_key: "ea236149c40d2007286311518ece9402"
server_key_iv: "7844c0245eb2e0c33efe4d8a99895d9c"
server_key_version: "encryptionkey@0"
I20260807 18:17:53.807855 15781 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786126328808105-19048-0/minicluster-data/master-0/wal/instance:
uuid: "2cd9df9d6aef47979a81b20edd4ad4c9"
format_stamp: "Formatted at 2026-08-07 18:17:53 on dist-test-slave-gkw7"
server_key: "ea236149c40d2007286311518ece9402"
server_key_iv: "7844c0245eb2e0c33efe4d8a99895d9c"
server_key_version: "encryptionkey@0"
I20260807 18:17:53.813375 15781 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260807 18:17:53.817205 15802 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:17:53.818706 15781 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:17:53.818908 15781 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786126328808105-19048-0/minicluster-data/master-0/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786126328808105-19048-0/minicluster-data/master-0/wal
uuid: "2cd9df9d6aef47979a81b20edd4ad4c9"
format_stamp: "Formatted at 2026-08-07 18:17:53 on dist-test-slave-gkw7"
server_key: "ea236149c40d2007286311518ece9402"
server_key_iv: "7844c0245eb2e0c33efe4d8a99895d9c"
server_key_version: "encryptionkey@0"
I20260807 18:17:53.819118 15781 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786126328808105-19048-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786126328808105-19048-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786126328808105-19048-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:17:53.835906 15781 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:17:53.840451 15781 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:17:53.840766 15781 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:17:53.860648 15781 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37807
I20260807 18:17:53.860648 15870 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37807 every 8 connection(s)
I20260807 18:17:53.862254 15781 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
I20260807 18:17:53.865440 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 15781
I20260807 18:17:53.865710 19048 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786126328808105-19048-0/minicluster-data/master-0/wal/instance
I20260807 18:17:53.866647 19048 external_mini_cluster.cc:1428] Setting key c0094b63ee270a2d02493b7ba4e4be28
I20260807 18:17:53.866822 15871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:17:53.881958 15871 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2cd9df9d6aef47979a81b20edd4ad4c9: Bootstrap starting.
Aug 07 18:17:53 dist-test-slave-gkw7 krb5kdc[15766](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126672, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260807 18:17:53.885607 15871 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2cd9df9d6aef47979a81b20edd4ad4c9: Neither blocks nor log segments found. Creating new log.
I20260807 18:17:53.886803 15871 log.cc:824] T 00000000000000000000000000000000 P 2cd9df9d6aef47979a81b20edd4ad4c9: Log is configured to *not* fsync() on all Append() calls
I20260807 18:17:53.889789 15871 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2cd9df9d6aef47979a81b20edd4ad4c9: No bootstrap required, opened a new log
I20260807 18:17:53.891350 15875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:53.868889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39871 (local address 127.0.0.1:37807)
0807 18:17:53.869900 (+ 1011us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:53.869912 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:53.869962 (+ 50us) server_negotiation.cc:407] Connection header received
0807 18:17:53.871723 (+ 1761us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:53.871746 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:53.872188 (+ 442us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:53.872569 (+ 381us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:17:53.874429 (+ 1860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:53.875791 (+ 1362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:53.877699 (+ 1908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:53.878404 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:53.878536 (+ 132us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:53.883665 (+ 5129us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:17:53.883674 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:17:53.883693 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:17:53.883736 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:17:53.886967 (+ 3231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:17:53.887805 (+ 838us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:17:53.887812 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:17:53.887822 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:17:53.888002 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:17:53.888510 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:17:53.888514 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:17:53.888516 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:17:53.889058 (+ 542us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:17:53.889396 (+ 338us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:53.889952 (+ 556us) server_negotiation.cc:299] Negotiation successful
0807 18:17:53.890323 (+ 371us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":779,"thread_start_us":447,"threads_started":1}
I20260807 18:17:53.895558 19048 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:17:53.899219 15871 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2cd9df9d6aef47979a81b20edd4ad4c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2cd9df9d6aef47979a81b20edd4ad4c9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37807 } }
I20260807 18:17:53.899775 15871 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2cd9df9d6aef47979a81b20edd4ad4c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:17:53.899912 15871 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2cd9df9d6aef47979a81b20edd4ad4c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2cd9df9d6aef47979a81b20edd4ad4c9, State: Initialized, Role: FOLLOWER
I20260807 18:17:53.900668 15871 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2cd9df9d6aef47979a81b20edd4ad4c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2cd9df9d6aef47979a81b20edd4ad4c9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37807 } }
I20260807 18:17:53.900993 15871 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2cd9df9d6aef47979a81b20edd4ad4c9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:17:53.901149 15871 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2cd9df9d6aef47979a81b20edd4ad4c9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:17:53.901333 15871 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2cd9df9d6aef47979a81b20edd4ad4c9 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:17:53.904839 15871 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2cd9df9d6aef47979a81b20edd4ad4c9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2cd9df9d6aef47979a81b20edd4ad4c9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37807 } }
I20260807 18:17:53.905504 15871 leader_election.cc:302] T 00000000000000000000000000000000 P 2cd9df9d6aef47979a81b20edd4ad4c9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2cd9df9d6aef47979a81b20edd4ad4c9; no voters:
I20260807 18:17:53.906245 15871 leader_election.cc:288] T 00000000000000000000000000000000 P 2cd9df9d6aef47979a81b20edd4ad4c9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:17:53.906368 15888 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2cd9df9d6aef47979a81b20edd4ad4c9 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:17:53.908002 15888 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2cd9df9d6aef47979a81b20edd4ad4c9 [term 1 LEADER]: Becoming Leader. State: Replica: 2cd9df9d6aef47979a81b20edd4ad4c9, State: Running, Role: LEADER
I20260807 18:17:53.908703 15888 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2cd9df9d6aef47979a81b20edd4ad4c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2cd9df9d6aef47979a81b20edd4ad4c9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37807 } }
I20260807 18:17:53.908883 15871 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2cd9df9d6aef47979a81b20edd4ad4c9 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:17:53.917637 15875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:53.901895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60187 (local address 127.0.0.1:37807)
0807 18:17:53.902144 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:53.902155 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:53.902685 (+ 530us) server_negotiation.cc:407] Connection header received
0807 18:17:53.902896 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:53.902901 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:53.902968 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:53.903057 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:17:53.904995 (+ 1938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:53.905954 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:53.907763 (+ 1809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:53.908491 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:53.908651 (+ 160us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:53.911558 (+ 2907us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:17:53.911563 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:17:53.911566 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:17:53.911599 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:17:53.914533 (+ 2934us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:17:53.915392 (+ 859us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:17:53.915398 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:17:53.915401 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:17:53.915528 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:17:53.915994 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:17:53.916003 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:17:53.916005 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:17:53.916309 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:17:53.916649 (+ 340us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:53.916985 (+ 336us) server_negotiation.cc:299] Negotiation successful
0807 18:17:53.917179 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260807 18:17:53.919982 15889 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2cd9df9d6aef47979a81b20edd4ad4c9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2cd9df9d6aef47979a81b20edd4ad4c9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2cd9df9d6aef47979a81b20edd4ad4c9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37807 } } }
I20260807 18:17:53.920344 15889 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2cd9df9d6aef47979a81b20edd4ad4c9 [sys.catalog]: This master's current role is: LEADER
I20260807 18:17:53.924638 15891 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2cd9df9d6aef47979a81b20edd4ad4c9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2cd9df9d6aef47979a81b20edd4ad4c9. Latest consensus state: current_term: 1 leader_uuid: "2cd9df9d6aef47979a81b20edd4ad4c9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2cd9df9d6aef47979a81b20edd4ad4c9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37807 } } }
I20260807 18:17:53.925043 15891 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2cd9df9d6aef47979a81b20edd4ad4c9 [sys.catalog]: This master's current role is: LEADER
I20260807 18:17:53.925849 15899 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:17:53.930302 15899 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:17:53.938664 15899 catalog_manager.cc:1347] Generated new cluster ID: 70d10eb554764101bca2ebc50d534c63
I20260807 18:17:53.938804 15899 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:17:53.961786 15899 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:17:53.963824 15899 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:17:53.977448 15899 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2cd9df9d6aef47979a81b20edd4ad4c9: Generated new TSK 0
I20260807 18:17:53.978482 15899 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:17:53.994508 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 15781
I20260807 18:17:53.996275 15865 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:17:54.091681 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 15781
W20260807 18:17:54.114857 15763 connection.cc:535] client connection to 127.0.0.1:37807 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37807): Connection reset by peer (error 104)
2026-08-07T18:17:54Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (3173 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786126328808105-19048-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:17:54 dist-test-slave-gkw7 krb5kdc[15916](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:17:54 dist-test-slave-gkw7 krb5kdc[15916](info): set up 2 sockets
Aug 07 18:17:54 dist-test-slave-gkw7 krb5kdc[15916](info): commencing operation
krb5kdc: starting...
W20260807 18:17:55.401049 19048 mini_kdc.cc:121] Time spent starting KDC: real 1.267s user 0.001s sys 0.044s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786126328808105-19048-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786126328808105-19048-0/krb5kdc/test-user.keytab.
Aug 07 18:17:55 dist-test-slave-gkw7 krb5kdc[15916](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126675, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:17:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:17:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786126328808105-19048-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786126328808105-19048-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 18:17:55.713449 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786126328808105-19048-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786126328808105-19048-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786126328808105-19048-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46009
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786126328808105-19048-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45327
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46009
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786126328808105-19048-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786126328808105-19048-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786126328808105-19048-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:17:56.067025 15939 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:17:56.067327 15939 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:17:56.067504 15939 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:17:56.074924 15939 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:17:56.075047 15939 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:17:56.075145 15939 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:17:56.075263 15939 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:17:56.075335 15939 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:17:56.086233 15939 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45327
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786126328808105-19048-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786126328808105-19048-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46009
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786126328808105-19048-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46009
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786126328808105-19048-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:17:56.087631 15939 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:17:56.089004 15939 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:17:56.101814 15950 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:17:56.101814 15947 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:17:56.101845 15948 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:17:56.103010 15939 server_base.cc:1034] running on GCE node
I20260807 18:17:56.104003 15939 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:17:56.105715 15939 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:17:56.106983 15939 hybrid_clock.cc:648] HybridClock initialized: now 1786126676106931 us; error 57 us; skew 500 ppm
I20260807 18:17:56.107409 15939 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:17:56 dist-test-slave-gkw7 krb5kdc[15916](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126676, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:17:56.111057 15939 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 18:17:56.112972 15939 webserver.cc:460] Webserver started at http://127.0.0.1:38109/ using document root <none> and password file <none>
I20260807 18:17:56.113880 15939 fs_manager.cc:359] Metadata directory not provided
I20260807 18:17:56.114050 15939 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:17:56.114483 15939 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:17:56.118078 15939 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786126328808105-19048-0/minicluster-data/master-0/data/instance:
uuid: "0718834a285f40d092b9b3ced8018071"
format_stamp: "Formatted at 2026-08-07 18:17:56 on dist-test-slave-gkw7"
server_key: "1e378eba770f53271096237b7ee895af"
server_key_iv: "7880a5a6107b39e0f62242f6a25c9b86"
server_key_version: "encryptionkey@0"
I20260807 18:17:56.119000 15939 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786126328808105-19048-0/minicluster-data/master-0/wal/instance:
uuid: "0718834a285f40d092b9b3ced8018071"
format_stamp: "Formatted at 2026-08-07 18:17:56 on dist-test-slave-gkw7"
server_key: "1e378eba770f53271096237b7ee895af"
server_key_iv: "7880a5a6107b39e0f62242f6a25c9b86"
server_key_version: "encryptionkey@0"
I20260807 18:17:56.124953 15939 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260807 18:17:56.129005 15963 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:17:56.130460 15939 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:17:56.130671 15939 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786126328808105-19048-0/minicluster-data/master-0/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786126328808105-19048-0/minicluster-data/master-0/wal
uuid: "0718834a285f40d092b9b3ced8018071"
format_stamp: "Formatted at 2026-08-07 18:17:56 on dist-test-slave-gkw7"
server_key: "1e378eba770f53271096237b7ee895af"
server_key_iv: "7880a5a6107b39e0f62242f6a25c9b86"
server_key_version: "encryptionkey@0"
I20260807 18:17:56.130908 15939 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786126328808105-19048-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786126328808105-19048-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786126328808105-19048-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:17:56.160946 15939 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:17:56.165748 15939 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:17:56.166091 15939 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:17:56.187505 15939 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46009
I20260807 18:17:56.187529 16034 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46009 every 8 connection(s)
I20260807 18:17:56.189119 15939 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
I20260807 18:17:56.193116 16035 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:17:56.196231 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 15939
I20260807 18:17:56.196452 19048 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786126328808105-19048-0/minicluster-data/master-0/wal/instance
I20260807 18:17:56.196960 19048 external_mini_cluster.cc:1428] Setting key 341da4905d25790d3abc095154c2bf85
I20260807 18:17:56.206964 16035 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0718834a285f40d092b9b3ced8018071: Bootstrap starting.
I20260807 18:17:56.211139 16035 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0718834a285f40d092b9b3ced8018071: Neither blocks nor log segments found. Creating new log.
I20260807 18:17:56.212563 16035 log.cc:824] T 00000000000000000000000000000000 P 0718834a285f40d092b9b3ced8018071: Log is configured to *not* fsync() on all Append() calls
I20260807 18:17:56.213373 16038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:56.199146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37427 (local address 127.0.0.1:46009)
0807 18:17:56.199985 (+ 839us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:56.200001 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:56.200051 (+ 50us) server_negotiation.cc:407] Connection header received
0807 18:17:56.202307 (+ 2256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:56.202340 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:56.202900 (+ 560us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:56.203379 (+ 479us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:17:56.203991 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:56.206079 (+ 2088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:56.207660 (+ 1581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:56.208767 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:56.208982 (+ 215us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:56.210354 (+ 1372us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:17:56.210364 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:17:56.210383 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:17:56.211123 (+ 740us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:17:56.211233 (+ 110us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:17:56.211271 (+ 38us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:17:56.211463 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:56.211738 (+ 275us) server_negotiation.cc:299] Negotiation successful
0807 18:17:56.211925 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":587,"thread_start_us":357,"threads_started":1}
I20260807 18:17:56.215947 16035 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0718834a285f40d092b9b3ced8018071: No bootstrap required, opened a new log
I20260807 18:17:56.217511 19048 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:17:56.224404 16035 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0718834a285f40d092b9b3ced8018071 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0718834a285f40d092b9b3ced8018071" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46009 } }
I20260807 18:17:56.224821 16035 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0718834a285f40d092b9b3ced8018071 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:17:56.224946 16035 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0718834a285f40d092b9b3ced8018071 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0718834a285f40d092b9b3ced8018071, State: Initialized, Role: FOLLOWER
I20260807 18:17:56.225662 16035 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0718834a285f40d092b9b3ced8018071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0718834a285f40d092b9b3ced8018071" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46009 } }
I20260807 18:17:56.225934 16035 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0718834a285f40d092b9b3ced8018071 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:17:56.226087 16035 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0718834a285f40d092b9b3ced8018071 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:17:56.226272 16035 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0718834a285f40d092b9b3ced8018071 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:17:56.228623 16035 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0718834a285f40d092b9b3ced8018071 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0718834a285f40d092b9b3ced8018071" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46009 } }
I20260807 18:17:56.229249 16035 leader_election.cc:302] T 00000000000000000000000000000000 P 0718834a285f40d092b9b3ced8018071 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0718834a285f40d092b9b3ced8018071; no voters:
I20260807 18:17:56.229789 16038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:56.222571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34947 (local address 127.0.0.1:46009)
0807 18:17:56.222834 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:56.222846 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:56.223182 (+ 336us) server_negotiation.cc:407] Connection header received
0807 18:17:56.223384 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:56.223388 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:56.223451 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:56.223537 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:17:56.224191 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:56.225213 (+ 1022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:56.226940 (+ 1727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:17:56.227509 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:17:56.227727 (+ 218us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:17:56.229002 (+ 1275us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:17:56.229009 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:17:56.229013 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:17:56.229034 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:17:56.229080 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:17:56.229101 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:17:56.229274 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:56.229381 (+ 107us) server_negotiation.cc:299] Negotiation successful
0807 18:17:56.229440 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260807 18:17:56.230094 16035 leader_election.cc:288] T 00000000000000000000000000000000 P 0718834a285f40d092b9b3ced8018071 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:17:56.230248 16046 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0718834a285f40d092b9b3ced8018071 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:17:56.231956 16046 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0718834a285f40d092b9b3ced8018071 [term 1 LEADER]: Becoming Leader. State: Replica: 0718834a285f40d092b9b3ced8018071, State: Running, Role: LEADER
I20260807 18:17:56.232493 16035 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0718834a285f40d092b9b3ced8018071 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:17:56.232609 16046 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0718834a285f40d092b9b3ced8018071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0718834a285f40d092b9b3ced8018071" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46009 } }
I20260807 18:17:56.236363 16047 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0718834a285f40d092b9b3ced8018071 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0718834a285f40d092b9b3ced8018071" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0718834a285f40d092b9b3ced8018071" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46009 } } }
I20260807 18:17:56.236826 16047 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0718834a285f40d092b9b3ced8018071 [sys.catalog]: This master's current role is: LEADER
I20260807 18:17:56.237777 16048 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0718834a285f40d092b9b3ced8018071 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0718834a285f40d092b9b3ced8018071. Latest consensus state: current_term: 1 leader_uuid: "0718834a285f40d092b9b3ced8018071" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0718834a285f40d092b9b3ced8018071" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46009 } } }
I20260807 18:17:56.238301 16048 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0718834a285f40d092b9b3ced8018071 [sys.catalog]: This master's current role is: LEADER
I20260807 18:17:56.243176 16056 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:17:56.248670 16056 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:17:56.259521 16056 catalog_manager.cc:1347] Generated new cluster ID: 86d42d4c082e417b97b573e23af3176f
I20260807 18:17:56.259708 16056 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:17:56.274272 16056 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:17:56.276185 16056 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:17:56.285609 16056 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0718834a285f40d092b9b3ced8018071: Generated new TSK 0
I20260807 18:17:56.286736 16056 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:17:56.300606 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 15939
W20260807 18:17:56.302333 16028 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:37427
W20260807 18:17:56.303216 19048 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 15939: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 18:17:56.303427 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 15939
W20260807 18:17:56.325242 15913 connection.cc:535] client connection to 127.0.0.1:46009 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:46009): Connection reset by peer (error 104)
2026-08-07T18:17:56Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (2211 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786126328808105-19048-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:17:56 dist-test-slave-gkw7 krb5kdc[16070](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:17:56 dist-test-slave-gkw7 krb5kdc[16070](info): set up 2 sockets
Aug 07 18:17:56 dist-test-slave-gkw7 krb5kdc[16070](info): commencing operation
krb5kdc: starting...
W20260807 18:17:57.632419 19048 mini_kdc.cc:121] Time spent starting KDC: real 1.286s user 0.004s sys 0.045s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786126328808105-19048-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786126328808105-19048-0/krb5kdc/test-user.keytab.
Aug 07 18:17:57 dist-test-slave-gkw7 krb5kdc[16070](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126677, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:17:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:17:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786126328808105-19048-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786126328808105-19048-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 18:17:57.961292 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786126328808105-19048-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786126328808105-19048-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786126328808105-19048-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46511
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786126328808105-19048-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34373
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46511
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786126328808105-19048-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786126328808105-19048-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786126328808105-19048-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:17:58.320086 16085 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:17:58.320395 16085 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:17:58.320567 16085 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:17:58.327879 16085 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:17:58.328003 16085 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:17:58.328133 16085 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:17:58.328279 16085 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:17:58.328342 16085 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:17:58.338913 16085 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34373
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786126328808105-19048-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786126328808105-19048-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46511
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786126328808105-19048-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46511
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786126328808105-19048-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:17:58.340372 16085 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:17:58.341818 16085 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:17:58.354249 16094 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:17:58.354377 16092 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:17:58.354277 16091 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:17:58.354923 16085 server_base.cc:1034] running on GCE node
I20260807 18:17:58.355739 16085 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:17:58.357256 16085 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:17:58.358510 16085 hybrid_clock.cc:648] HybridClock initialized: now 1786126678358445 us; error 79 us; skew 500 ppm
I20260807 18:17:58.358932 16085 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:17:58 dist-test-slave-gkw7 krb5kdc[16070](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126678, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:17:58.362365 16085 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 18:17:58.363926 16085 webserver.cc:460] Webserver started at http://127.0.0.1:36893/ using document root <none> and password file <none>
I20260807 18:17:58.364645 16085 fs_manager.cc:359] Metadata directory not provided
I20260807 18:17:58.364760 16085 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:17:58.365029 16085 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:17:58.368191 16085 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786126328808105-19048-0/minicluster-data/master-0/data/instance:
uuid: "dd89bd2d7151474bb8efa00343945842"
format_stamp: "Formatted at 2026-08-07 18:17:58 on dist-test-slave-gkw7"
server_key: "2fe5cd2fcd3e86d7bdfd588ce79182bc"
server_key_iv: "950b30f979428192660dca7c1f503bc9"
server_key_version: "encryptionkey@0"
I20260807 18:17:58.368917 16085 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786126328808105-19048-0/minicluster-data/master-0/wal/instance:
uuid: "dd89bd2d7151474bb8efa00343945842"
format_stamp: "Formatted at 2026-08-07 18:17:58 on dist-test-slave-gkw7"
server_key: "2fe5cd2fcd3e86d7bdfd588ce79182bc"
server_key_iv: "950b30f979428192660dca7c1f503bc9"
server_key_version: "encryptionkey@0"
I20260807 18:17:58.374418 16085 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.002s
I20260807 18:17:58.378137 16104 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:17:58.379654 16085 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260807 18:17:58.379809 16085 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786126328808105-19048-0/minicluster-data/master-0/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786126328808105-19048-0/minicluster-data/master-0/wal
uuid: "dd89bd2d7151474bb8efa00343945842"
format_stamp: "Formatted at 2026-08-07 18:17:58 on dist-test-slave-gkw7"
server_key: "2fe5cd2fcd3e86d7bdfd588ce79182bc"
server_key_iv: "950b30f979428192660dca7c1f503bc9"
server_key_version: "encryptionkey@0"
I20260807 18:17:58.379987 16085 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786126328808105-19048-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786126328808105-19048-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786126328808105-19048-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:17:58.386346 16085 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:17:58.390444 16085 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:17:58.390712 16085 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:17:58.410844 16085 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46511
I20260807 18:17:58.410885 16190 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46511 every 8 connection(s)
I20260807 18:17:58.412410 16085 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
I20260807 18:17:58.414661 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 16085
I20260807 18:17:58.414844 19048 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786126328808105-19048-0/minicluster-data/master-0/wal/instance
I20260807 18:17:58.415331 19048 external_mini_cluster.cc:1428] Setting key 05cfe705e714acfd97d772a6cdbba896
I20260807 18:17:58.416934 16191 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:17:58.424636 16196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:58.417350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58585 (local address 127.0.0.1:46511)
0807 18:17:58.418270 (+ 920us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:58.418287 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:58.418340 (+ 53us) server_negotiation.cc:407] Connection header received
0807 18:17:58.420400 (+ 2060us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:58.420431 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:58.420941 (+ 510us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:58.421397 (+ 456us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:17:58.422488 (+ 1091us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:17:58.422498 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:17:58.422523 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:17:58.422540 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:17:58.422638 (+ 98us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:17:58.422667 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:17:58.422917 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:58.423066 (+ 149us) server_negotiation.cc:299] Negotiation successful
0807 18:17:58.423284 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":649,"thread_start_us":422,"threads_started":1}
I20260807 18:17:58.429010 19048 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:17:58.432392 16191 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dd89bd2d7151474bb8efa00343945842: Bootstrap starting.
I20260807 18:17:58.436800 16191 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dd89bd2d7151474bb8efa00343945842: Neither blocks nor log segments found. Creating new log.
I20260807 18:17:58.437769 16196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:17:58.435053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45737 (local address 127.0.0.1:46511)
0807 18:17:58.435246 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0807 18:17:58.435252 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:17:58.435618 (+ 366us) server_negotiation.cc:407] Connection header received
0807 18:17:58.435787 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:17:58.435793 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:17:58.435869 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:17:58.435998 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:17:58.437052 (+ 1054us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:17:58.437062 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:17:58.437071 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:17:58.437087 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:17:58.437135 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:17:58.437156 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:17:58.437325 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0807 18:17:58.437387 (+ 62us) server_negotiation.cc:299] Negotiation successful
0807 18:17:58.437453 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260807 18:17:58.438330 16191 log.cc:824] T 00000000000000000000000000000000 P dd89bd2d7151474bb8efa00343945842: Log is configured to *not* fsync() on all Append() calls
I20260807 18:17:58.441311 16191 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dd89bd2d7151474bb8efa00343945842: No bootstrap required, opened a new log
I20260807 18:17:58.448972 16191 raft_consensus.cc:348] T 00000000000000000000000000000000 P dd89bd2d7151474bb8efa00343945842 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd89bd2d7151474bb8efa00343945842" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46511 } }
I20260807 18:17:58.449445 16191 raft_consensus.cc:374] T 00000000000000000000000000000000 P dd89bd2d7151474bb8efa00343945842 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:17:58.449596 16191 raft_consensus.cc:729] T 00000000000000000000000000000000 P dd89bd2d7151474bb8efa00343945842 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd89bd2d7151474bb8efa00343945842, State: Initialized, Role: FOLLOWER
I20260807 18:17:58.450204 16191 consensus_queue.cc:260] T 00000000000000000000000000000000 P dd89bd2d7151474bb8efa00343945842 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd89bd2d7151474bb8efa00343945842" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46511 } }
I20260807 18:17:58.450475 16191 raft_consensus.cc:388] T 00000000000000000000000000000000 P dd89bd2d7151474bb8efa00343945842 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:17:58.450652 16191 raft_consensus.cc:482] T 00000000000000000000000000000000 P dd89bd2d7151474bb8efa00343945842 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:17:58.450858 16191 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dd89bd2d7151474bb8efa00343945842 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:17:58.453260 16191 raft_consensus.cc:504] T 00000000000000000000000000000000 P dd89bd2d7151474bb8efa00343945842 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd89bd2d7151474bb8efa00343945842" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46511 } }
I20260807 18:17:58.453948 16191 leader_election.cc:302] T 00000000000000000000000000000000 P dd89bd2d7151474bb8efa00343945842 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dd89bd2d7151474bb8efa00343945842; no voters:
I20260807 18:17:58.454579 16191 leader_election.cc:288] T 00000000000000000000000000000000 P dd89bd2d7151474bb8efa00343945842 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:17:58.454722 16204 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dd89bd2d7151474bb8efa00343945842 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:17:58.455507 16204 raft_consensus.cc:686] T 00000000000000000000000000000000 P dd89bd2d7151474bb8efa00343945842 [term 1 LEADER]: Becoming Leader. State: Replica: dd89bd2d7151474bb8efa00343945842, State: Running, Role: LEADER
I20260807 18:17:58.456097 16204 consensus_queue.cc:237] T 00000000000000000000000000000000 P dd89bd2d7151474bb8efa00343945842 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd89bd2d7151474bb8efa00343945842" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46511 } }
I20260807 18:17:58.456866 16191 sys_catalog.cc:564] T 00000000000000000000000000000000 P dd89bd2d7151474bb8efa00343945842 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:17:58.459115 16206 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd89bd2d7151474bb8efa00343945842 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dd89bd2d7151474bb8efa00343945842. Latest consensus state: current_term: 1 leader_uuid: "dd89bd2d7151474bb8efa00343945842" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd89bd2d7151474bb8efa00343945842" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46511 } } }
I20260807 18:17:58.459331 16205 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd89bd2d7151474bb8efa00343945842 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dd89bd2d7151474bb8efa00343945842" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd89bd2d7151474bb8efa00343945842" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46511 } } }
I20260807 18:17:58.459597 16205 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd89bd2d7151474bb8efa00343945842 [sys.catalog]: This master's current role is: LEADER
I20260807 18:17:58.459597 16206 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd89bd2d7151474bb8efa00343945842 [sys.catalog]: This master's current role is: LEADER
I20260807 18:17:58.460767 16211 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:17:58.467628 16211 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:17:58.477311 16211 catalog_manager.cc:1347] Generated new cluster ID: b4b014882edc4523be7e35a923598781
I20260807 18:17:58.477450 16211 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:17:58.491921 16211 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:17:58.492448 16211 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:17:58.502588 16211 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dd89bd2d7151474bb8efa00343945842: Generated new TSK 0
I20260807 18:17:58.503820 16211 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:17:58.510387 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 16085
W20260807 18:17:58.511564 16182 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:58585
W20260807 18:17:58.512337 19048 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 16085: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 18:17:58.512506 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 16085
2026-08-07T18:17:58Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (2211 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786126328808105-19048-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:17:58 dist-test-slave-gkw7 krb5kdc[16228](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:17:58 dist-test-slave-gkw7 krb5kdc[16228](info): set up 2 sockets
Aug 07 18:17:58 dist-test-slave-gkw7 krb5kdc[16228](info): commencing operation
krb5kdc: starting...
W20260807 18:17:59.820742 19048 mini_kdc.cc:121] Time spent starting KDC: real 1.263s user 0.003s sys 0.043s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786126328808105-19048-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786126328808105-19048-0/krb5kdc/test-user.keytab.
Aug 07 18:18:00 dist-test-slave-gkw7 krb5kdc[16228](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126680, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:18:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:18:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786126328808105-19048-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786126328808105-19048-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 18:18:00.159128 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786126328808105-19048-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786126328808105-19048-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786126328808105-19048-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39021
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786126328808105-19048-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45221
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39021
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786126328808105-19048-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786126328808105-19048-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786126328808105-19048-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:18:00.518695 16244 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:18:00.519166 16244 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:18:00.519387 16244 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:18:00.527501 16244 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:18:00.527621 16244 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:18:00.527741 16244 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:18:00.527841 16244 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:18:00.527930 16244 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:18:00.538355 16244 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45221
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786126328808105-19048-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786126328808105-19048-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39021
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786126328808105-19048-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39021
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786126328808105-19048-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:18:00.539798 16244 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:18:00.541306 16244 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:18:00.553829 16253 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:18:00.554292 16256 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:18:00.554394 16254 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:18:00.555156 16244 server_base.cc:1034] running on GCE node
I20260807 18:18:00.555991 16244 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:18:00.557639 16244 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:18:00.558892 16244 hybrid_clock.cc:648] HybridClock initialized: now 1786126680558849 us; error 42 us; skew 500 ppm
I20260807 18:18:00.559321 16244 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:18:00 dist-test-slave-gkw7 krb5kdc[16228](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126680, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:18:00.570398 16244 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 18:18:00.572207 16244 webserver.cc:460] Webserver started at http://127.0.0.1:40981/ using document root <none> and password file <none>
I20260807 18:18:00.572993 16244 fs_manager.cc:359] Metadata directory not provided
I20260807 18:18:00.573159 16244 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:18:00.573511 16244 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:18:00.576701 16244 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786126328808105-19048-0/minicluster-data/master-0/data/instance:
uuid: "64c0a0ab6b554f219804c67b5eaafc20"
format_stamp: "Formatted at 2026-08-07 18:18:00 on dist-test-slave-gkw7"
server_key: "e6c52f3be6e2f70406ec240b04cd9655"
server_key_iv: "7230a2b8987f8282de130d533f945bdb"
server_key_version: "encryptionkey@0"
I20260807 18:18:00.577592 16244 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786126328808105-19048-0/minicluster-data/master-0/wal/instance:
uuid: "64c0a0ab6b554f219804c67b5eaafc20"
format_stamp: "Formatted at 2026-08-07 18:18:00 on dist-test-slave-gkw7"
server_key: "e6c52f3be6e2f70406ec240b04cd9655"
server_key_iv: "7230a2b8987f8282de130d533f945bdb"
server_key_version: "encryptionkey@0"
I20260807 18:18:00.583271 16244 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260807 18:18:00.587049 16265 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:18:00.588529 16244 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 18:18:00.588739 16244 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786126328808105-19048-0/minicluster-data/master-0/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786126328808105-19048-0/minicluster-data/master-0/wal
uuid: "64c0a0ab6b554f219804c67b5eaafc20"
format_stamp: "Formatted at 2026-08-07 18:18:00 on dist-test-slave-gkw7"
server_key: "e6c52f3be6e2f70406ec240b04cd9655"
server_key_iv: "7230a2b8987f8282de130d533f945bdb"
server_key_version: "encryptionkey@0"
I20260807 18:18:00.588956 16244 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786126328808105-19048-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786126328808105-19048-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786126328808105-19048-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:18:00.610941 16244 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:18:00.615796 16244 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:18:00.616117 16244 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:18:00.636688 16244 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39021
I20260807 18:18:00.636752 16340 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39021 every 8 connection(s)
I20260807 18:18:00.638201 16244 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
I20260807 18:18:00.642382 16341 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:18:00.642541 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 16244
I20260807 18:18:00.642765 19048 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786126328808105-19048-0/minicluster-data/master-0/wal/instance
I20260807 18:18:00.643213 19048 external_mini_cluster.cc:1428] Setting key ccef0511ccc8dd2e2cc60e212ee7bc7f
I20260807 18:18:00.658704 16341 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 64c0a0ab6b554f219804c67b5eaafc20: Bootstrap starting.
Aug 07 18:18:00 dist-test-slave-gkw7 krb5kdc[16228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126680, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260807 18:18:00.663007 16341 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 64c0a0ab6b554f219804c67b5eaafc20: Neither blocks nor log segments found. Creating new log.
I20260807 18:18:00.664254 16341 log.cc:824] T 00000000000000000000000000000000 P 64c0a0ab6b554f219804c67b5eaafc20: Log is configured to *not* fsync() on all Append() calls
I20260807 18:18:00.667935 16341 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 64c0a0ab6b554f219804c67b5eaafc20: No bootstrap required, opened a new log
I20260807 18:18:00.668996 16344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:00.645047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38371 (local address 127.0.0.1:39021)
0807 18:18:00.646033 (+ 986us) server_negotiation.cc:206] Beginning negotiation
0807 18:18:00.646049 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 18:18:00.646103 (+ 54us) server_negotiation.cc:407] Connection header received
0807 18:18:00.648145 (+ 2042us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:18:00.648177 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:18:00.648764 (+ 587us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:18:00.649260 (+ 496us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:18:00.650842 (+ 1582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:00.653073 (+ 2231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:00.654475 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:00.655748 (+ 1273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:00.655986 (+ 238us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:18:00.661209 (+ 5223us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:18:00.661218 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:18:00.661235 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:18:00.661281 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:18:00.664629 (+ 3348us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:18:00.665568 (+ 939us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:18:00.665577 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:18:00.665588 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:18:00.665792 (+ 204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:18:00.666256 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:18:00.666260 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:18:00.666262 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:18:00.666787 (+ 525us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:18:00.667006 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0807 18:18:00.667536 (+ 530us) server_negotiation.cc:299] Negotiation successful
0807 18:18:00.667884 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":706,"thread_start_us":416,"threads_started":1}
I20260807 18:18:00.672987 19048 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:18:00.677369 16341 raft_consensus.cc:348] T 00000000000000000000000000000000 P 64c0a0ab6b554f219804c67b5eaafc20 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64c0a0ab6b554f219804c67b5eaafc20" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39021 } }
I20260807 18:18:00.677923 16341 raft_consensus.cc:374] T 00000000000000000000000000000000 P 64c0a0ab6b554f219804c67b5eaafc20 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:18:00.678051 16341 raft_consensus.cc:729] T 00000000000000000000000000000000 P 64c0a0ab6b554f219804c67b5eaafc20 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64c0a0ab6b554f219804c67b5eaafc20, State: Initialized, Role: FOLLOWER
I20260807 18:18:00.678792 16341 consensus_queue.cc:260] T 00000000000000000000000000000000 P 64c0a0ab6b554f219804c67b5eaafc20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64c0a0ab6b554f219804c67b5eaafc20" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39021 } }
I20260807 18:18:00.679092 16341 raft_consensus.cc:388] T 00000000000000000000000000000000 P 64c0a0ab6b554f219804c67b5eaafc20 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:18:00.679286 16341 raft_consensus.cc:482] T 00000000000000000000000000000000 P 64c0a0ab6b554f219804c67b5eaafc20 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:18:00.679523 16341 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 64c0a0ab6b554f219804c67b5eaafc20 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:18:00.683128 16341 raft_consensus.cc:504] T 00000000000000000000000000000000 P 64c0a0ab6b554f219804c67b5eaafc20 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64c0a0ab6b554f219804c67b5eaafc20" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39021 } }
I20260807 18:18:00.683840 16341 leader_election.cc:302] T 00000000000000000000000000000000 P 64c0a0ab6b554f219804c67b5eaafc20 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 64c0a0ab6b554f219804c67b5eaafc20; no voters:
I20260807 18:18:00.684593 16341 leader_election.cc:288] T 00000000000000000000000000000000 P 64c0a0ab6b554f219804c67b5eaafc20 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:18:00.684731 16355 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 64c0a0ab6b554f219804c67b5eaafc20 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:18:00.686172 16355 raft_consensus.cc:686] T 00000000000000000000000000000000 P 64c0a0ab6b554f219804c67b5eaafc20 [term 1 LEADER]: Becoming Leader. State: Replica: 64c0a0ab6b554f219804c67b5eaafc20, State: Running, Role: LEADER
I20260807 18:18:00.686941 16355 consensus_queue.cc:237] T 00000000000000000000000000000000 P 64c0a0ab6b554f219804c67b5eaafc20 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64c0a0ab6b554f219804c67b5eaafc20" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39021 } }
I20260807 18:18:00.687266 16341 sys_catalog.cc:564] T 00000000000000000000000000000000 P 64c0a0ab6b554f219804c67b5eaafc20 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:18:00.690598 16357 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64c0a0ab6b554f219804c67b5eaafc20 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 64c0a0ab6b554f219804c67b5eaafc20. Latest consensus state: current_term: 1 leader_uuid: "64c0a0ab6b554f219804c67b5eaafc20" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64c0a0ab6b554f219804c67b5eaafc20" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39021 } } }
I20260807 18:18:00.691157 16357 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64c0a0ab6b554f219804c67b5eaafc20 [sys.catalog]: This master's current role is: LEADER
I20260807 18:18:00.691874 16356 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64c0a0ab6b554f219804c67b5eaafc20 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "64c0a0ab6b554f219804c67b5eaafc20" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64c0a0ab6b554f219804c67b5eaafc20" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39021 } } }
I20260807 18:18:00.692276 16356 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64c0a0ab6b554f219804c67b5eaafc20 [sys.catalog]: This master's current role is: LEADER
I20260807 18:18:00.695525 16367 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:18:00.700803 16344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:00.677483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45729 (local address 127.0.0.1:39021)
0807 18:18:00.677693 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0807 18:18:00.677702 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:18:00.679263 (+ 1561us) server_negotiation.cc:407] Connection header received
0807 18:18:00.679460 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:18:00.679468 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:18:00.679576 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:18:00.679716 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:18:00.682117 (+ 2401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:00.682838 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:00.684912 (+ 2074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:00.686099 (+ 1187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:00.686297 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:18:00.690360 (+ 4063us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:18:00.690368 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:18:00.690375 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:18:00.690435 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:18:00.695617 (+ 5182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:18:00.696734 (+ 1117us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:18:00.696740 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:18:00.696744 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:18:00.696936 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:18:00.697468 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:18:00.697473 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:18:00.697477 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:18:00.698123 (+ 646us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:18:00.698532 (+ 409us) server_negotiation.cc:1035] Waiting for connection context
0807 18:18:00.700055 (+ 1523us) server_negotiation.cc:299] Negotiation successful
0807 18:18:00.700326 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260807 18:18:00.702948 16367 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:18:00.712765 16367 catalog_manager.cc:1347] Generated new cluster ID: 24e91605cf4348beb18bba10c5fc540f
I20260807 18:18:00.712903 16367 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:18:00.724404 16367 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:18:00.726042 16367 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:18:00.733294 16367 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 64c0a0ab6b554f219804c67b5eaafc20: Generated new TSK 0
I20260807 18:18:00.734318 16367 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:18:00.739845 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 16244
I20260807 18:18:00.741248 16334 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:18:00.831911 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 16244
2026-08-07T18:18:00Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (2312 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786126328808105-19048-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:18:00 dist-test-slave-gkw7 krb5kdc[16384](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:18:00 dist-test-slave-gkw7 krb5kdc[16384](info): set up 2 sockets
Aug 07 18:18:00 dist-test-slave-gkw7 krb5kdc[16384](info): commencing operation
krb5kdc: starting...
W20260807 18:18:02.157706 19048 mini_kdc.cc:121] Time spent starting KDC: real 1.288s user 0.004s sys 0.041s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786126328808105-19048-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786126328808105-19048-0/krb5kdc/test-user.keytab.
Aug 07 18:18:02 dist-test-slave-gkw7 krb5kdc[16384](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126682, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:18:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:18:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786126328808105-19048-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786126328808105-19048-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 18:18:02.497815 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786126328808105-19048-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786126328808105-19048-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786126328808105-19048-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36215
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786126328808105-19048-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36215
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786126328808105-19048-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786126328808105-19048-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786126328808105-19048-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:18:02.853909 16401 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:18:02.854218 16401 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:18:02.854346 16401 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:18:02.861737 16401 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:18:02.861912 16401 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:18:02.862020 16401 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:18:02.862133 16401 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:18:02.862222 16401 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:18:02.872624 16401 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786126328808105-19048-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786126328808105-19048-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36215
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786126328808105-19048-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36215
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786126328808105-19048-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:18:02.874163 16401 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:18:02.875532 16401 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:18:02.887850 16413 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:18:02.887923 16410 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:18:02.887821 16408 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:18:02.888712 16401 server_base.cc:1034] running on GCE node
I20260807 18:18:02.889612 16401 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:18:02.891103 16401 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:18:02.892313 16401 hybrid_clock.cc:648] HybridClock initialized: now 1786126682892260 us; error 44 us; skew 500 ppm
I20260807 18:18:02.892683 16401 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:18:02 dist-test-slave-gkw7 krb5kdc[16384](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126682, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:18:02.896039 16401 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 18:18:02.897817 16401 webserver.cc:460] Webserver started at http://127.0.0.1:44675/ using document root <none> and password file <none>
I20260807 18:18:02.898582 16401 fs_manager.cc:359] Metadata directory not provided
I20260807 18:18:02.898731 16401 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:18:02.899073 16401 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:18:02.902333 16401 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786126328808105-19048-0/minicluster-data/master-0/data/instance:
uuid: "6127c32cebca4ccda2a684e631ee7341"
format_stamp: "Formatted at 2026-08-07 18:18:02 on dist-test-slave-gkw7"
server_key: "4fbcb1a47667d5cb321ada89a24394ef"
server_key_iv: "497642d06f1be5133d1075c423de98a3"
server_key_version: "encryptionkey@0"
I20260807 18:18:02.903200 16401 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786126328808105-19048-0/minicluster-data/master-0/wal/instance:
uuid: "6127c32cebca4ccda2a684e631ee7341"
format_stamp: "Formatted at 2026-08-07 18:18:02 on dist-test-slave-gkw7"
server_key: "4fbcb1a47667d5cb321ada89a24394ef"
server_key_iv: "497642d06f1be5133d1075c423de98a3"
server_key_version: "encryptionkey@0"
I20260807 18:18:02.908782 16401 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 18:18:02.912616 16421 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:18:02.914285 16401 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:18:02.914492 16401 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786126328808105-19048-0/minicluster-data/master-0/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786126328808105-19048-0/minicluster-data/master-0/wal
uuid: "6127c32cebca4ccda2a684e631ee7341"
format_stamp: "Formatted at 2026-08-07 18:18:02 on dist-test-slave-gkw7"
server_key: "4fbcb1a47667d5cb321ada89a24394ef"
server_key_iv: "497642d06f1be5133d1075c423de98a3"
server_key_version: "encryptionkey@0"
I20260807 18:18:02.914705 16401 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786126328808105-19048-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786126328808105-19048-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786126328808105-19048-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:18:02.936656 16401 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:18:02.941416 16401 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:18:02.941794 16401 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:18:02.961763 16401 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36215
I20260807 18:18:02.961808 16499 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36215 every 8 connection(s)
I20260807 18:18:02.963300 16401 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
I20260807 18:18:02.967300 16500 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:18:02.971598 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 16401
I20260807 18:18:02.971810 19048 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786126328808105-19048-0/minicluster-data/master-0/wal/instance
I20260807 18:18:02.972412 19048 external_mini_cluster.cc:1428] Setting key 65969b8e5c4dffe11830f0a38869bec5
I20260807 18:18:02.980646 16500 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6127c32cebca4ccda2a684e631ee7341: Bootstrap starting.
I20260807 18:18:02.984218 16500 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6127c32cebca4ccda2a684e631ee7341: Neither blocks nor log segments found. Creating new log.
I20260807 18:18:02.985455 16500 log.cc:824] T 00000000000000000000000000000000 P 6127c32cebca4ccda2a684e631ee7341: Log is configured to *not* fsync() on all Append() calls
I20260807 18:18:02.986380 16506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:02.974436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54875 (local address 127.0.0.1:36215)
0807 18:18:02.975282 (+ 846us) server_negotiation.cc:206] Beginning negotiation
0807 18:18:02.975294 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 18:18:02.975338 (+ 44us) server_negotiation.cc:407] Connection header received
0807 18:18:02.977089 (+ 1751us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:18:02.977117 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:18:02.977653 (+ 536us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:18:02.978113 (+ 460us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:18:02.978735 (+ 622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:02.980062 (+ 1327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:02.981485 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:02.982295 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:02.982507 (+ 212us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:18:02.983731 (+ 1224us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:18:02.983739 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:18:02.983760 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:18:02.984315 (+ 555us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:18:02.984437 (+ 122us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:18:02.984478 (+ 41us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:18:02.984618 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0807 18:18:02.984869 (+ 251us) server_negotiation.cc:299] Negotiation successful
0807 18:18:02.985141 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":619,"thread_start_us":390,"threads_started":1}
I20260807 18:18:02.988543 16500 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6127c32cebca4ccda2a684e631ee7341: No bootstrap required, opened a new log
I20260807 18:18:02.990195 19048 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:18:02.997290 16500 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6127c32cebca4ccda2a684e631ee7341 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6127c32cebca4ccda2a684e631ee7341" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36215 } }
I20260807 18:18:02.997882 16500 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6127c32cebca4ccda2a684e631ee7341 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:18:02.997996 16500 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6127c32cebca4ccda2a684e631ee7341 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6127c32cebca4ccda2a684e631ee7341, State: Initialized, Role: FOLLOWER
I20260807 18:18:02.998695 16500 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6127c32cebca4ccda2a684e631ee7341 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6127c32cebca4ccda2a684e631ee7341" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36215 } }
I20260807 18:18:02.999037 16500 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6127c32cebca4ccda2a684e631ee7341 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:18:02.999213 16500 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6127c32cebca4ccda2a684e631ee7341 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:18:02.999428 16500 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6127c32cebca4ccda2a684e631ee7341 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:18:03.001585 16506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:02.994842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54655 (local address 127.0.0.1:36215)
0807 18:18:02.995065 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0807 18:18:02.995072 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:18:02.995387 (+ 315us) server_negotiation.cc:407] Connection header received
0807 18:18:02.995597 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:18:02.995601 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:18:02.995666 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:18:02.995743 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:18:02.996366 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:02.997061 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:02.998972 (+ 1911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:02.999559 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:02.999672 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:18:03.000810 (+ 1138us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:18:03.000815 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:18:03.000818 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:18:03.000835 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:18:03.000872 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:18:03.000891 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:18:03.000966 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0807 18:18:03.001122 (+ 156us) server_negotiation.cc:299] Negotiation successful
0807 18:18:03.001196 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260807 18:18:03.002866 16500 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6127c32cebca4ccda2a684e631ee7341 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6127c32cebca4ccda2a684e631ee7341" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36215 } }
I20260807 18:18:03.003536 16500 leader_election.cc:302] T 00000000000000000000000000000000 P 6127c32cebca4ccda2a684e631ee7341 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6127c32cebca4ccda2a684e631ee7341; no voters:
I20260807 18:18:03.004204 16500 leader_election.cc:288] T 00000000000000000000000000000000 P 6127c32cebca4ccda2a684e631ee7341 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:18:03.004336 16516 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6127c32cebca4ccda2a684e631ee7341 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:18:03.005091 16516 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6127c32cebca4ccda2a684e631ee7341 [term 1 LEADER]: Becoming Leader. State: Replica: 6127c32cebca4ccda2a684e631ee7341, State: Running, Role: LEADER
I20260807 18:18:03.005757 16516 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6127c32cebca4ccda2a684e631ee7341 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6127c32cebca4ccda2a684e631ee7341" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36215 } }
I20260807 18:18:03.006603 16500 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6127c32cebca4ccda2a684e631ee7341 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:18:03.008731 16521 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6127c32cebca4ccda2a684e631ee7341 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6127c32cebca4ccda2a684e631ee7341. Latest consensus state: current_term: 1 leader_uuid: "6127c32cebca4ccda2a684e631ee7341" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6127c32cebca4ccda2a684e631ee7341" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36215 } } }
I20260807 18:18:03.008607 16517 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6127c32cebca4ccda2a684e631ee7341 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6127c32cebca4ccda2a684e631ee7341" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6127c32cebca4ccda2a684e631ee7341" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36215 } } }
I20260807 18:18:03.009261 16517 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6127c32cebca4ccda2a684e631ee7341 [sys.catalog]: This master's current role is: LEADER
I20260807 18:18:03.010802 16521 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6127c32cebca4ccda2a684e631ee7341 [sys.catalog]: This master's current role is: LEADER
I20260807 18:18:03.010957 16525 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:18:03.017421 16525 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:18:03.027928 16538 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260807 18:18:03.029520 16525 catalog_manager.cc:1347] Generated new cluster ID: 359130cf9c8e40a69df99d630c998fc5
I20260807 18:18:03.029668 16525 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:18:03.037976 16525 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:18:03.039605 16525 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:18:03.045694 16525 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6127c32cebca4ccda2a684e631ee7341: Generated new TSK 0
I20260807 18:18:03.046600 16525 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:18:03.053617 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 16401
W20260807 18:18:03.054870 16490 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:54875
W20260807 18:18:03.055662 19048 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 16401: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 18:18:03.055831 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 16401
2026-08-07T18:18:03Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (2232 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786126328808105-19048-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:18:03 dist-test-slave-gkw7 krb5kdc[16541](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:18:03 dist-test-slave-gkw7 krb5kdc[16541](info): set up 2 sockets
Aug 07 18:18:03 dist-test-slave-gkw7 krb5kdc[16541](info): commencing operation
krb5kdc: starting...
W20260807 18:18:04.398241 19048 mini_kdc.cc:121] Time spent starting KDC: real 1.296s user 0.002s sys 0.045s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786126328808105-19048-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786126328808105-19048-0/krb5kdc/test-user.keytab.
Aug 07 18:18:04 dist-test-slave-gkw7 krb5kdc[16541](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126684, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:18:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:18:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786126328808105-19048-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786126328808105-19048-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 18:18:04.737448 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786126328808105-19048-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786126328808105-19048-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786126328808105-19048-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40613
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786126328808105-19048-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37647
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40613
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786126328808105-19048-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786126328808105-19048-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786126328808105-19048-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:18:05.103878 16557 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:18:05.104188 16557 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:18:05.104307 16557 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:18:05.111775 16557 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:18:05.111896 16557 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:18:05.111943 16557 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:18:05.112006 16557 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:18:05.112051 16557 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:18:05.122535 16557 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37647
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786126328808105-19048-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786126328808105-19048-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40613
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786126328808105-19048-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40613
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786126328808105-19048-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:18:05.123893 16557 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:18:05.125326 16557 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:18:05.138412 16569 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:18:05.138495 16565 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:18:05.138559 16564 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:18:05.139518 16557 server_base.cc:1034] running on GCE node
I20260807 18:18:05.140360 16557 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:18:05.142023 16557 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:18:05.143289 16557 hybrid_clock.cc:648] HybridClock initialized: now 1786126685143237 us; error 52 us; skew 500 ppm
I20260807 18:18:05.143692 16557 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:18:05 dist-test-slave-gkw7 krb5kdc[16541](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126685, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:18:05.147198 16557 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 18:18:05.148833 16557 webserver.cc:460] Webserver started at http://127.0.0.1:39339/ using document root <none> and password file <none>
I20260807 18:18:05.149648 16557 fs_manager.cc:359] Metadata directory not provided
I20260807 18:18:05.149809 16557 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:18:05.150139 16557 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:18:05.153265 16557 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786126328808105-19048-0/minicluster-data/master-0/data/instance:
uuid: "c8b4679f99184ccf94802bb877712e21"
format_stamp: "Formatted at 2026-08-07 18:18:05 on dist-test-slave-gkw7"
server_key: "9b735edbc5380db4a6fe46dcb3f4f62d"
server_key_iv: "e23485f5a0d19dfc02c446cfdd7eed2c"
server_key_version: "encryptionkey@0"
I20260807 18:18:05.154131 16557 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786126328808105-19048-0/minicluster-data/master-0/wal/instance:
uuid: "c8b4679f99184ccf94802bb877712e21"
format_stamp: "Formatted at 2026-08-07 18:18:05 on dist-test-slave-gkw7"
server_key: "9b735edbc5380db4a6fe46dcb3f4f62d"
server_key_iv: "e23485f5a0d19dfc02c446cfdd7eed2c"
server_key_version: "encryptionkey@0"
I20260807 18:18:05.159652 16557 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 18:18:05.163689 16579 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:18:05.165316 16557 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 18:18:05.165516 16557 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786126328808105-19048-0/minicluster-data/master-0/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786126328808105-19048-0/minicluster-data/master-0/wal
uuid: "c8b4679f99184ccf94802bb877712e21"
format_stamp: "Formatted at 2026-08-07 18:18:05 on dist-test-slave-gkw7"
server_key: "9b735edbc5380db4a6fe46dcb3f4f62d"
server_key_iv: "e23485f5a0d19dfc02c446cfdd7eed2c"
server_key_version: "encryptionkey@0"
I20260807 18:18:05.165855 16557 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786126328808105-19048-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786126328808105-19048-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786126328808105-19048-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:18:05.172021 16557 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:18:05.176213 16557 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:18:05.176527 16557 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:18:05.196893 16557 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40613
I20260807 18:18:05.196954 16646 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40613 every 8 connection(s)
I20260807 18:18:05.198432 16557 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
I20260807 18:18:05.201834 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 16557
I20260807 18:18:05.202060 19048 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786126328808105-19048-0/minicluster-data/master-0/wal/instance
I20260807 18:18:05.202544 19048 external_mini_cluster.cc:1428] Setting key b15974f1ef12279e8cd46cf699dedc07
I20260807 18:18:05.202852 16648 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:18:05.212173 16652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:05.204398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51871 (local address 127.0.0.1:40613)
0807 18:18:05.205241 (+ 843us) server_negotiation.cc:206] Beginning negotiation
0807 18:18:05.205257 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 18:18:05.205309 (+ 52us) server_negotiation.cc:407] Connection header received
0807 18:18:05.207934 (+ 2625us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:18:05.207969 (+ 35us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:18:05.208463 (+ 494us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:18:05.208975 (+ 512us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:18:05.210084 (+ 1109us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:18:05.210094 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:18:05.210116 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:18:05.210133 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:18:05.210219 (+ 86us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:18:05.210251 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:18:05.210496 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0807 18:18:05.210659 (+ 163us) server_negotiation.cc:299] Negotiation successful
0807 18:18:05.210873 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":581,"thread_start_us":345,"threads_started":1}
I20260807 18:18:05.216434 19048 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:18:05.218334 16648 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c8b4679f99184ccf94802bb877712e21: Bootstrap starting.
I20260807 18:18:05.222235 16648 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c8b4679f99184ccf94802bb877712e21: Neither blocks nor log segments found. Creating new log.
I20260807 18:18:05.223796 16652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:05.220967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41895 (local address 127.0.0.1:40613)
0807 18:18:05.221184 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0807 18:18:05.221192 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:18:05.221713 (+ 521us) server_negotiation.cc:407] Connection header received
0807 18:18:05.221878 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:18:05.221885 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:18:05.221958 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:18:05.222082 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:18:05.223065 (+ 983us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:18:05.223070 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:18:05.223075 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:18:05.223093 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:18:05.223145 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:18:05.223167 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:18:05.223291 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0807 18:18:05.223377 (+ 86us) server_negotiation.cc:299] Negotiation successful
0807 18:18:05.223446 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 18:18:05.223794 16648 log.cc:824] T 00000000000000000000000000000000 P c8b4679f99184ccf94802bb877712e21: Log is configured to *not* fsync() on all Append() calls
I20260807 18:18:05.227042 16648 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c8b4679f99184ccf94802bb877712e21: No bootstrap required, opened a new log
I20260807 18:18:05.234633 16648 raft_consensus.cc:348] T 00000000000000000000000000000000 P c8b4679f99184ccf94802bb877712e21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8b4679f99184ccf94802bb877712e21" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40613 } }
I20260807 18:18:05.235049 16648 raft_consensus.cc:374] T 00000000000000000000000000000000 P c8b4679f99184ccf94802bb877712e21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:18:05.235163 16648 raft_consensus.cc:729] T 00000000000000000000000000000000 P c8b4679f99184ccf94802bb877712e21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8b4679f99184ccf94802bb877712e21, State: Initialized, Role: FOLLOWER
I20260807 18:18:05.235916 16648 consensus_queue.cc:260] T 00000000000000000000000000000000 P c8b4679f99184ccf94802bb877712e21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8b4679f99184ccf94802bb877712e21" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40613 } }
I20260807 18:18:05.236225 16648 raft_consensus.cc:388] T 00000000000000000000000000000000 P c8b4679f99184ccf94802bb877712e21 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:18:05.236389 16648 raft_consensus.cc:482] T 00000000000000000000000000000000 P c8b4679f99184ccf94802bb877712e21 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:18:05.236575 16648 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c8b4679f99184ccf94802bb877712e21 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:18:05.238919 16648 raft_consensus.cc:504] T 00000000000000000000000000000000 P c8b4679f99184ccf94802bb877712e21 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8b4679f99184ccf94802bb877712e21" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40613 } }
I20260807 18:18:05.239471 16648 leader_election.cc:302] T 00000000000000000000000000000000 P c8b4679f99184ccf94802bb877712e21 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c8b4679f99184ccf94802bb877712e21; no voters:
I20260807 18:18:05.240099 16648 leader_election.cc:288] T 00000000000000000000000000000000 P c8b4679f99184ccf94802bb877712e21 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:18:05.240304 16661 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c8b4679f99184ccf94802bb877712e21 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:18:05.241326 16661 raft_consensus.cc:686] T 00000000000000000000000000000000 P c8b4679f99184ccf94802bb877712e21 [term 1 LEADER]: Becoming Leader. State: Replica: c8b4679f99184ccf94802bb877712e21, State: Running, Role: LEADER
I20260807 18:18:05.242030 16661 consensus_queue.cc:237] T 00000000000000000000000000000000 P c8b4679f99184ccf94802bb877712e21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8b4679f99184ccf94802bb877712e21" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40613 } }
I20260807 18:18:05.242709 16648 sys_catalog.cc:564] T 00000000000000000000000000000000 P c8b4679f99184ccf94802bb877712e21 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:18:05.245281 16666 sys_catalog.cc:455] T 00000000000000000000000000000000 P c8b4679f99184ccf94802bb877712e21 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c8b4679f99184ccf94802bb877712e21. Latest consensus state: current_term: 1 leader_uuid: "c8b4679f99184ccf94802bb877712e21" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8b4679f99184ccf94802bb877712e21" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40613 } } }
I20260807 18:18:05.245787 16666 sys_catalog.cc:458] T 00000000000000000000000000000000 P c8b4679f99184ccf94802bb877712e21 [sys.catalog]: This master's current role is: LEADER
I20260807 18:18:05.246503 16665 sys_catalog.cc:455] T 00000000000000000000000000000000 P c8b4679f99184ccf94802bb877712e21 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c8b4679f99184ccf94802bb877712e21" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8b4679f99184ccf94802bb877712e21" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40613 } } }
I20260807 18:18:05.248281 16665 sys_catalog.cc:458] T 00000000000000000000000000000000 P c8b4679f99184ccf94802bb877712e21 [sys.catalog]: This master's current role is: LEADER
I20260807 18:18:05.252537 16676 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:18:05.258812 16676 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:18:05.272425 16676 catalog_manager.cc:1347] Generated new cluster ID: 281f8110e4384f61b7a5a55575f3ade0
I20260807 18:18:05.272557 16676 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:18:05.311159 16676 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:18:05.311621 16676 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:18:05.319451 16676 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c8b4679f99184ccf94802bb877712e21: Generated new TSK 0
I20260807 18:18:05.320634 16676 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:18:05.336457 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 16557
W20260807 18:18:05.337709 16642 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:51871
W20260807 18:18:05.338464 19048 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 16557: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 18:18:05.338637 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 16557
2026-08-07T18:18:05Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (2281 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786126328808105-19048-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:18:05 dist-test-slave-gkw7 krb5kdc[16692](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:18:05 dist-test-slave-gkw7 krb5kdc[16692](info): set up 2 sockets
Aug 07 18:18:05 dist-test-slave-gkw7 krb5kdc[16692](info): commencing operation
krb5kdc: starting...
W20260807 18:18:06.693997 19048 mini_kdc.cc:121] Time spent starting KDC: real 1.309s user 0.001s sys 0.046s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786126328808105-19048-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786126328808105-19048-0/krb5kdc/test-user.keytab.
Aug 07 18:18:06 dist-test-slave-gkw7 krb5kdc[16692](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126686, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:18:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:18:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786126328808105-19048-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786126328808105-19048-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 18:18:07.042007 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786126328808105-19048-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786126328808105-19048-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786126328808105-19048-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37251
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786126328808105-19048-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37885
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37251
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786126328808105-19048-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786126328808105-19048-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786126328808105-19048-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:18:07.410832 16706 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:18:07.411212 16706 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:18:07.411376 16706 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:18:07.419194 16706 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:18:07.419363 16706 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:18:07.419467 16706 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:18:07.419582 16706 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:18:07.419656 16706 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:18:07.430339 16706 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37885
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786126328808105-19048-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786126328808105-19048-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37251
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786126328808105-19048-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37251
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786126328808105-19048-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:18:07.431778 16706 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:18:07.433199 16706 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:18:07.445959 16712 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:18:07.446015 16714 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:18:07.446108 16716 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:18:07.447599 16706 server_base.cc:1034] running on GCE node
I20260807 18:18:07.448549 16706 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:18:07.450201 16706 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:18:07.451452 16706 hybrid_clock.cc:648] HybridClock initialized: now 1786126687451406 us; error 43 us; skew 500 ppm
I20260807 18:18:07.451824 16706 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:18:07 dist-test-slave-gkw7 krb5kdc[16692](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126687, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:18:07.455456 16706 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 18:18:07.457217 16706 webserver.cc:460] Webserver started at http://127.0.0.1:39361/ using document root <none> and password file <none>
I20260807 18:18:07.458050 16706 fs_manager.cc:359] Metadata directory not provided
I20260807 18:18:07.458226 16706 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:18:07.458575 16706 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:18:07.462028 16706 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786126328808105-19048-0/minicluster-data/master-0/data/instance:
uuid: "0f3213555ce6435b8802a5f04925f445"
format_stamp: "Formatted at 2026-08-07 18:18:07 on dist-test-slave-gkw7"
server_key: "10e0d945f760d14c3ac830ffbbbec4e9"
server_key_iv: "eb42c9dbd513c2704673d3356958549c"
server_key_version: "encryptionkey@0"
I20260807 18:18:07.462908 16706 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786126328808105-19048-0/minicluster-data/master-0/wal/instance:
uuid: "0f3213555ce6435b8802a5f04925f445"
format_stamp: "Formatted at 2026-08-07 18:18:07 on dist-test-slave-gkw7"
server_key: "10e0d945f760d14c3ac830ffbbbec4e9"
server_key_iv: "eb42c9dbd513c2704673d3356958549c"
server_key_version: "encryptionkey@0"
I20260807 18:18:07.468681 16706 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260807 18:18:07.472559 16724 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:18:07.474049 16706 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:18:07.474299 16706 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786126328808105-19048-0/minicluster-data/master-0/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786126328808105-19048-0/minicluster-data/master-0/wal
uuid: "0f3213555ce6435b8802a5f04925f445"
format_stamp: "Formatted at 2026-08-07 18:18:07 on dist-test-slave-gkw7"
server_key: "10e0d945f760d14c3ac830ffbbbec4e9"
server_key_iv: "eb42c9dbd513c2704673d3356958549c"
server_key_version: "encryptionkey@0"
I20260807 18:18:07.474512 16706 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786126328808105-19048-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786126328808105-19048-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786126328808105-19048-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:18:07.495119 16706 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:18:07.499805 16706 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:18:07.500131 16706 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:18:07.519976 16706 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37251
I20260807 18:18:07.520010 16796 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37251 every 8 connection(s)
I20260807 18:18:07.521509 16706 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
I20260807 18:18:07.525401 16797 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:18:07.527038 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 16706
I20260807 18:18:07.527247 19048 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786126328808105-19048-0/minicluster-data/master-0/wal/instance
I20260807 18:18:07.527849 19048 external_mini_cluster.cc:1428] Setting key 3acaf36fdd4afb6610e21ad59194eec3
I20260807 18:18:07.539541 16797 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0f3213555ce6435b8802a5f04925f445: Bootstrap starting.
Aug 07 18:18:07 dist-test-slave-gkw7 krb5kdc[16692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126686, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260807 18:18:07.543401 16797 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0f3213555ce6435b8802a5f04925f445: Neither blocks nor log segments found. Creating new log.
I20260807 18:18:07.544921 16797 log.cc:824] T 00000000000000000000000000000000 P 0f3213555ce6435b8802a5f04925f445: Log is configured to *not* fsync() on all Append() calls
I20260807 18:18:07.548601 16797 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0f3213555ce6435b8802a5f04925f445: No bootstrap required, opened a new log
I20260807 18:18:07.553421 16801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:07.530097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.125:57465 (local address 127.0.0.1:37251)
0807 18:18:07.530922 (+ 825us) server_negotiation.cc:206] Beginning negotiation
0807 18:18:07.530935 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 18:18:07.530984 (+ 49us) server_negotiation.cc:407] Connection header received
0807 18:18:07.532770 (+ 1786us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:18:07.532795 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:18:07.533227 (+ 432us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:18:07.533687 (+ 460us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:18:07.535356 (+ 1669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:07.536824 (+ 1468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:07.538388 (+ 1564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:07.539129 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:07.539341 (+ 212us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:18:07.544835 (+ 5494us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:18:07.544844 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:18:07.544860 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:18:07.544905 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:18:07.548817 (+ 3912us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:18:07.549911 (+ 1094us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:18:07.549918 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:18:07.549925 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:18:07.550119 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:18:07.550653 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:18:07.550656 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:18:07.550658 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:18:07.551220 (+ 562us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:18:07.551476 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0807 18:18:07.551995 (+ 519us) server_negotiation.cc:299] Negotiation successful
0807 18:18:07.552345 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":588,"thread_start_us":360,"threads_started":1}
I20260807 18:18:07.557781 19048 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:18:07.557981 16797 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0f3213555ce6435b8802a5f04925f445 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f3213555ce6435b8802a5f04925f445" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37251 } }
I20260807 18:18:07.558398 16797 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0f3213555ce6435b8802a5f04925f445 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:18:07.558493 16797 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0f3213555ce6435b8802a5f04925f445 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f3213555ce6435b8802a5f04925f445, State: Initialized, Role: FOLLOWER
I20260807 18:18:07.559157 16797 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0f3213555ce6435b8802a5f04925f445 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f3213555ce6435b8802a5f04925f445" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37251 } }
I20260807 18:18:07.559439 16797 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0f3213555ce6435b8802a5f04925f445 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:18:07.559566 16797 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0f3213555ce6435b8802a5f04925f445 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:18:07.559764 16797 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0f3213555ce6435b8802a5f04925f445 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:18:07.563707 16797 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0f3213555ce6435b8802a5f04925f445 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f3213555ce6435b8802a5f04925f445" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37251 } }
I20260807 18:18:07.564456 16797 leader_election.cc:302] T 00000000000000000000000000000000 P 0f3213555ce6435b8802a5f04925f445 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0f3213555ce6435b8802a5f04925f445; no voters:
I20260807 18:18:07.565244 16797 leader_election.cc:288] T 00000000000000000000000000000000 P 0f3213555ce6435b8802a5f04925f445 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:18:07.565358 16808 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0f3213555ce6435b8802a5f04925f445 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:18:07.567126 16808 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0f3213555ce6435b8802a5f04925f445 [term 1 LEADER]: Becoming Leader. State: Replica: 0f3213555ce6435b8802a5f04925f445, State: Running, Role: LEADER
I20260807 18:18:07.567734 16797 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0f3213555ce6435b8802a5f04925f445 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:18:07.567889 16808 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0f3213555ce6435b8802a5f04925f445 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f3213555ce6435b8802a5f04925f445" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37251 } }
I20260807 18:18:07.579746 16810 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0f3213555ce6435b8802a5f04925f445 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0f3213555ce6435b8802a5f04925f445. Latest consensus state: current_term: 1 leader_uuid: "0f3213555ce6435b8802a5f04925f445" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f3213555ce6435b8802a5f04925f445" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37251 } } }
I20260807 18:18:07.579668 16809 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0f3213555ce6435b8802a5f04925f445 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0f3213555ce6435b8802a5f04925f445" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f3213555ce6435b8802a5f04925f445" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37251 } } }
I20260807 18:18:07.580250 16810 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0f3213555ce6435b8802a5f04925f445 [sys.catalog]: This master's current role is: LEADER
I20260807 18:18:07.580264 16809 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0f3213555ce6435b8802a5f04925f445 [sys.catalog]: This master's current role is: LEADER
W20260807 18:18:07.586230 16828 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0f3213555ce6435b8802a5f04925f445: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 18:18:07.586447 16828 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 18:18:07.587891 16801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:07.563434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.125:49799 (local address 127.0.0.1:37251)
0807 18:18:07.563662 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0807 18:18:07.563671 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:18:07.564412 (+ 741us) server_negotiation.cc:407] Connection header received
0807 18:18:07.564588 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:18:07.564592 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:18:07.564697 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:18:07.564810 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:18:07.566902 (+ 2092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:07.568246 (+ 1344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:07.570526 (+ 2280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:07.571541 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:07.571708 (+ 167us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:18:07.578075 (+ 6367us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:18:07.578082 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:18:07.578087 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:18:07.578134 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:18:07.583039 (+ 4905us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:18:07.584454 (+ 1415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:18:07.584462 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:18:07.584469 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:18:07.584660 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:18:07.585422 (+ 762us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:18:07.585430 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:18:07.585436 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:18:07.586049 (+ 613us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:18:07.586414 (+ 365us) server_negotiation.cc:1035] Waiting for connection context
0807 18:18:07.587272 (+ 858us) server_negotiation.cc:299] Negotiation successful
0807 18:18:07.587474 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":43}
I20260807 18:18:07.590884 16830 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:18:07.592483 16830 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:18:07.601356 16830 catalog_manager.cc:1347] Generated new cluster ID: 36737f8ecc4149a0bd8d60fbf3e24edc
I20260807 18:18:07.601480 16830 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:18:07.616722 16830 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:18:07.618705 16830 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:18:07.629716 16830 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0f3213555ce6435b8802a5f04925f445: Generated new TSK 0
I20260807 18:18:07.630963 16830 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:18:07.646117 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 16706
I20260807 18:18:07.647620 16791 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:18:07.743770 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 16706
W20260807 18:18:07.770243 16690 connection.cc:535] client connection to 127.0.0.1:37251 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37251): Connection reset by peer (error 104)
2026-08-07T18:18:07Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (2405 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786126328808105-19048-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:18:07 dist-test-slave-gkw7 krb5kdc[16839](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:18:07 dist-test-slave-gkw7 krb5kdc[16839](info): set up 2 sockets
Aug 07 18:18:07 dist-test-slave-gkw7 krb5kdc[16839](info): commencing operation
krb5kdc: starting...
W20260807 18:18:09.091121 19048 mini_kdc.cc:121] Time spent starting KDC: real 1.301s user 0.000s sys 0.044s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786126328808105-19048-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786126328808105-19048-0/krb5kdc/test-user.keytab.
Aug 07 18:18:09 dist-test-slave-gkw7 krb5kdc[16839](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126689, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:18:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:18:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786126328808105-19048-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786126328808105-19048-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 18:18:09.429382 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786126328808105-19048-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786126328808105-19048-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786126328808105-19048-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34143
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786126328808105-19048-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36753
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34143
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786126328808105-19048-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786126328808105-19048-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786126328808105-19048-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:18:09.779809 16856 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:18:09.780108 16856 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:18:09.780283 16856 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:18:09.787627 16856 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:18:09.787743 16856 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:18:09.787863 16856 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:18:09.787981 16856 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:18:09.788050 16856 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:18:09.797968 16856 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36753
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786126328808105-19048-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786126328808105-19048-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34143
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786126328808105-19048-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34143
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786126328808105-19048-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:18:09.799335 16856 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:18:09.800721 16856 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:18:09.812935 16865 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:18:09.813040 16861 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:18:09.813016 16863 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:18:09.814347 16856 server_base.cc:1034] running on GCE node
I20260807 18:18:09.815182 16856 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:18:09.816635 16856 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:18:09.817873 16856 hybrid_clock.cc:648] HybridClock initialized: now 1786126689817824 us; error 42 us; skew 500 ppm
I20260807 18:18:09.818266 16856 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:18:09 dist-test-slave-gkw7 krb5kdc[16839](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126689, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:18:09.821619 16856 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 18:18:09.823207 16856 webserver.cc:460] Webserver started at http://127.0.0.1:33925/ using document root <none> and password file <none>
I20260807 18:18:09.823938 16856 fs_manager.cc:359] Metadata directory not provided
I20260807 18:18:09.824105 16856 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:18:09.824525 16856 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:18:09.827688 16856 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786126328808105-19048-0/minicluster-data/master-0/data/instance:
uuid: "2dd221e8228349feb782b7951abbaa6d"
format_stamp: "Formatted at 2026-08-07 18:18:09 on dist-test-slave-gkw7"
server_key: "08ae80cc2bedebd0518a167f0d3a3786"
server_key_iv: "17045732db7da357e54ae7983a603f00"
server_key_version: "encryptionkey@0"
I20260807 18:18:09.828512 16856 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786126328808105-19048-0/minicluster-data/master-0/wal/instance:
uuid: "2dd221e8228349feb782b7951abbaa6d"
format_stamp: "Formatted at 2026-08-07 18:18:09 on dist-test-slave-gkw7"
server_key: "08ae80cc2bedebd0518a167f0d3a3786"
server_key_iv: "17045732db7da357e54ae7983a603f00"
server_key_version: "encryptionkey@0"
I20260807 18:18:09.834157 16856 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260807 18:18:09.837872 16874 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:18:09.839354 16856 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 18:18:09.839594 16856 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786126328808105-19048-0/minicluster-data/master-0/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786126328808105-19048-0/minicluster-data/master-0/wal
uuid: "2dd221e8228349feb782b7951abbaa6d"
format_stamp: "Formatted at 2026-08-07 18:18:09 on dist-test-slave-gkw7"
server_key: "08ae80cc2bedebd0518a167f0d3a3786"
server_key_iv: "17045732db7da357e54ae7983a603f00"
server_key_version: "encryptionkey@0"
I20260807 18:18:09.839838 16856 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786126328808105-19048-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786126328808105-19048-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786126328808105-19048-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:18:09.855683 16856 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:18:09.860086 16856 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:18:09.860383 16856 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:18:09.882853 16856 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34143
I20260807 18:18:09.882875 16948 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34143 every 8 connection(s)
I20260807 18:18:09.884380 16856 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
I20260807 18:18:09.888208 16949 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:18:09.891748 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 16856
I20260807 18:18:09.891945 19048 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786126328808105-19048-0/minicluster-data/master-0/wal/instance
I20260807 18:18:09.892547 19048 external_mini_cluster.cc:1428] Setting key 2284aae601c7c1fa7ba03c5527101dac
I20260807 18:18:09.901026 16949 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2dd221e8228349feb782b7951abbaa6d: Bootstrap starting.
I20260807 18:18:09.904829 16949 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2dd221e8228349feb782b7951abbaa6d: Neither blocks nor log segments found. Creating new log.
I20260807 18:18:09.906047 16949 log.cc:824] T 00000000000000000000000000000000 P 2dd221e8228349feb782b7951abbaa6d: Log is configured to *not* fsync() on all Append() calls
I20260807 18:18:09.906895 16952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:09.894451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.125:51675 (local address 127.0.0.1:34143)
0807 18:18:09.895255 (+ 804us) server_negotiation.cc:206] Beginning negotiation
0807 18:18:09.895267 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 18:18:09.895311 (+ 44us) server_negotiation.cc:407] Connection header received
0807 18:18:09.897016 (+ 1705us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:18:09.897040 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:18:09.897476 (+ 436us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:18:09.897917 (+ 441us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:18:09.898538 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:09.899883 (+ 1345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:09.901871 (+ 1988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:09.902546 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:09.902712 (+ 166us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:18:09.904156 (+ 1444us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:18:09.904163 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:18:09.904176 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:18:09.904720 (+ 544us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:18:09.904827 (+ 107us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:18:09.904852 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:18:09.905088 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0807 18:18:09.905426 (+ 338us) server_negotiation.cc:299] Negotiation successful
0807 18:18:09.905675 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":601,"thread_start_us":390,"threads_started":1}
I20260807 18:18:09.909031 16949 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2dd221e8228349feb782b7951abbaa6d: No bootstrap required, opened a new log
I20260807 18:18:09.910876 19048 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:18:09.919648 16949 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2dd221e8228349feb782b7951abbaa6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2dd221e8228349feb782b7951abbaa6d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34143 } }
I20260807 18:18:09.920130 16949 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2dd221e8228349feb782b7951abbaa6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:18:09.920243 16949 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2dd221e8228349feb782b7951abbaa6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2dd221e8228349feb782b7951abbaa6d, State: Initialized, Role: FOLLOWER
I20260807 18:18:09.920928 16949 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2dd221e8228349feb782b7951abbaa6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2dd221e8228349feb782b7951abbaa6d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34143 } }
I20260807 18:18:09.921264 16949 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2dd221e8228349feb782b7951abbaa6d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:18:09.921432 16949 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2dd221e8228349feb782b7951abbaa6d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:18:09.921674 16949 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2dd221e8228349feb782b7951abbaa6d [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:18:09.924034 16952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:09.916616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.125:59425 (local address 127.0.0.1:34143)
0807 18:18:09.916815 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0807 18:18:09.916822 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:18:09.917399 (+ 577us) server_negotiation.cc:407] Connection header received
0807 18:18:09.917659 (+ 260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:18:09.917663 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:18:09.917733 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:18:09.917816 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:18:09.918451 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:09.919215 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:09.921202 (+ 1987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:09.921897 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:09.922002 (+ 105us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:18:09.923225 (+ 1223us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:18:09.923229 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:18:09.923232 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:18:09.923249 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:18:09.923308 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:18:09.923323 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:18:09.923409 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0807 18:18:09.923636 (+ 227us) server_negotiation.cc:299] Negotiation successful
0807 18:18:09.923702 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260807 18:18:09.924752 16949 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2dd221e8228349feb782b7951abbaa6d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2dd221e8228349feb782b7951abbaa6d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34143 } }
I20260807 18:18:09.925293 16949 leader_election.cc:302] T 00000000000000000000000000000000 P 2dd221e8228349feb782b7951abbaa6d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2dd221e8228349feb782b7951abbaa6d; no voters:
I20260807 18:18:09.925946 16949 leader_election.cc:288] T 00000000000000000000000000000000 P 2dd221e8228349feb782b7951abbaa6d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:18:09.926092 16964 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2dd221e8228349feb782b7951abbaa6d [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:18:09.926793 16964 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2dd221e8228349feb782b7951abbaa6d [term 1 LEADER]: Becoming Leader. State: Replica: 2dd221e8228349feb782b7951abbaa6d, State: Running, Role: LEADER
I20260807 18:18:09.927314 16964 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2dd221e8228349feb782b7951abbaa6d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2dd221e8228349feb782b7951abbaa6d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34143 } }
I20260807 18:18:09.928121 16949 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2dd221e8228349feb782b7951abbaa6d [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:18:09.931666 16967 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2dd221e8228349feb782b7951abbaa6d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2dd221e8228349feb782b7951abbaa6d. Latest consensus state: current_term: 1 leader_uuid: "2dd221e8228349feb782b7951abbaa6d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2dd221e8228349feb782b7951abbaa6d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34143 } } }
I20260807 18:18:09.932132 16967 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2dd221e8228349feb782b7951abbaa6d [sys.catalog]: This master's current role is: LEADER
I20260807 18:18:09.932953 16966 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2dd221e8228349feb782b7951abbaa6d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2dd221e8228349feb782b7951abbaa6d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2dd221e8228349feb782b7951abbaa6d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34143 } } }
I20260807 18:18:09.933349 16966 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2dd221e8228349feb782b7951abbaa6d [sys.catalog]: This master's current role is: LEADER
I20260807 18:18:09.934024 16976 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:18:09.939308 16976 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:18:09.949177 16976 catalog_manager.cc:1347] Generated new cluster ID: 48aa142f755f468cbb388580315dbc1e
I20260807 18:18:09.949302 16976 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:18:09.981009 16976 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:18:09.982599 16976 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:18:09.989562 16976 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2dd221e8228349feb782b7951abbaa6d: Generated new TSK 0
I20260807 18:18:09.990499 16976 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:18:10.010406 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 16856
W20260807 18:18:10.011724 16943 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.125:51675
W20260807 18:18:10.012526 19048 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 16856: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 18:18:10.012678 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 16856
W20260807 18:18:10.037158 16835 connection.cc:535] client connection to 127.0.0.1:34143 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34143): Connection reset by peer (error 104)
2026-08-07T18:18:10Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (2266 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786126328808105-19048-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:18:10 dist-test-slave-gkw7 krb5kdc[16994](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:18:10 dist-test-slave-gkw7 krb5kdc[16994](info): set up 2 sockets
Aug 07 18:18:10 dist-test-slave-gkw7 krb5kdc[16994](info): commencing operation
krb5kdc: starting...
W20260807 18:18:11.414417 19048 mini_kdc.cc:121] Time spent starting KDC: real 1.357s user 0.004s sys 0.042s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786126328808105-19048-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786126328808105-19048-0/krb5kdc/test-user.keytab.
Aug 07 18:18:11 dist-test-slave-gkw7 krb5kdc[16994](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126691, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:18:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:18:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786126328808105-19048-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786126328808105-19048-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 18:18:11.760660 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786126328808105-19048-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786126328808105-19048-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786126328808105-19048-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43783
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786126328808105-19048-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45151
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43783
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786126328808105-19048-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786126328808105-19048-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786126328808105-19048-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:18:12.118624 17013 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:18:12.118945 17013 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:18:12.119050 17013 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:18:12.126363 17013 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:18:12.126474 17013 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:18:12.126520 17013 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:18:12.126582 17013 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:18:12.126627 17013 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:18:12.136816 17013 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45151
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786126328808105-19048-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786126328808105-19048-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43783
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786126328808105-19048-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43783
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786126328808105-19048-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:18:12.138195 17013 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:18:12.139474 17013 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:18:12.152199 17019 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:18:12.152259 17020 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:18:12.152472 17024 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:18:12.153319 17013 server_base.cc:1034] running on GCE node
I20260807 18:18:12.154310 17013 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:18:12.155836 17013 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:18:12.157075 17013 hybrid_clock.cc:648] HybridClock initialized: now 1786126692157024 us; error 35 us; skew 500 ppm
I20260807 18:18:12.157421 17013 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:18:12 dist-test-slave-gkw7 krb5kdc[16994](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126692, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:18:12.165277 17013 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 18:18:12.167183 17013 webserver.cc:460] Webserver started at http://127.0.0.1:38007/ using document root <none> and password file <none>
I20260807 18:18:12.167939 17013 fs_manager.cc:359] Metadata directory not provided
I20260807 18:18:12.168089 17013 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:18:12.168454 17013 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:18:12.171761 17013 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786126328808105-19048-0/minicluster-data/master-0/data/instance:
uuid: "a561166c17cd4434b43162aca4ff7471"
format_stamp: "Formatted at 2026-08-07 18:18:12 on dist-test-slave-gkw7"
server_key: "0a607d3fada5dba22526f84ab413960b"
server_key_iv: "3d0d4c2875a1e0e0bd14bdfd4e931113"
server_key_version: "encryptionkey@0"
I20260807 18:18:12.172585 17013 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786126328808105-19048-0/minicluster-data/master-0/wal/instance:
uuid: "a561166c17cd4434b43162aca4ff7471"
format_stamp: "Formatted at 2026-08-07 18:18:12 on dist-test-slave-gkw7"
server_key: "0a607d3fada5dba22526f84ab413960b"
server_key_iv: "3d0d4c2875a1e0e0bd14bdfd4e931113"
server_key_version: "encryptionkey@0"
I20260807 18:18:12.178337 17013 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260807 18:18:12.182294 17036 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:18:12.183912 17013 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 18:18:12.184211 17013 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786126328808105-19048-0/minicluster-data/master-0/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786126328808105-19048-0/minicluster-data/master-0/wal
uuid: "a561166c17cd4434b43162aca4ff7471"
format_stamp: "Formatted at 2026-08-07 18:18:12 on dist-test-slave-gkw7"
server_key: "0a607d3fada5dba22526f84ab413960b"
server_key_iv: "3d0d4c2875a1e0e0bd14bdfd4e931113"
server_key_version: "encryptionkey@0"
I20260807 18:18:12.184435 17013 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786126328808105-19048-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786126328808105-19048-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786126328808105-19048-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:18:12.190624 17013 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:18:12.195359 17013 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:18:12.195711 17013 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:18:12.217192 17013 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43783
I20260807 18:18:12.217243 17103 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43783 every 8 connection(s)
I20260807 18:18:12.218812 17013 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
I20260807 18:18:12.222823 17104 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:18:12.225293 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 17013
I20260807 18:18:12.225512 19048 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786126328808105-19048-0/minicluster-data/master-0/wal/instance
I20260807 18:18:12.226085 19048 external_mini_cluster.cc:1428] Setting key 204a5715878ff1880f0cd2609e39bc21
I20260807 18:18:12.235066 17107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:12.228263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.125:60477 (local address 127.0.0.1:43783)
0807 18:18:12.229299 (+ 1036us) server_negotiation.cc:206] Beginning negotiation
0807 18:18:12.229313 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 18:18:12.229360 (+ 47us) server_negotiation.cc:407] Connection header received
0807 18:18:12.231272 (+ 1912us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:18:12.231298 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:18:12.231712 (+ 414us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:18:12.232105 (+ 393us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:18:12.233319 (+ 1214us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:18:12.233326 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:18:12.233340 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:18:12.233356 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:18:12.233425 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:18:12.233447 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:18:12.233709 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0807 18:18:12.233815 (+ 106us) server_negotiation.cc:299] Negotiation successful
0807 18:18:12.234077 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":798,"thread_start_us":517,"threads_started":1}
I20260807 18:18:12.235503 17104 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a561166c17cd4434b43162aca4ff7471: Bootstrap starting.
I20260807 18:18:12.238932 17104 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a561166c17cd4434b43162aca4ff7471: Neither blocks nor log segments found. Creating new log.
I20260807 18:18:12.239331 19048 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:18:12.240340 17104 log.cc:824] T 00000000000000000000000000000000 P a561166c17cd4434b43162aca4ff7471: Log is configured to *not* fsync() on all Append() calls
I20260807 18:18:12.244192 17104 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a561166c17cd4434b43162aca4ff7471: No bootstrap required, opened a new log
I20260807 18:18:12.247447 17107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:12.244745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.125:34425 (local address 127.0.0.1:43783)
0807 18:18:12.245000 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0807 18:18:12.245010 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:18:12.245441 (+ 431us) server_negotiation.cc:407] Connection header received
0807 18:18:12.245652 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:18:12.245656 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:18:12.245710 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:18:12.245795 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:18:12.246756 (+ 961us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:18:12.246760 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:18:12.246763 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:18:12.246778 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:18:12.246817 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:18:12.246836 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:18:12.246931 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0807 18:18:12.247071 (+ 140us) server_negotiation.cc:299] Negotiation successful
0807 18:18:12.247123 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260807 18:18:12.253033 17104 raft_consensus.cc:348] T 00000000000000000000000000000000 P a561166c17cd4434b43162aca4ff7471 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a561166c17cd4434b43162aca4ff7471" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43783 } }
I20260807 18:18:12.253481 17104 raft_consensus.cc:374] T 00000000000000000000000000000000 P a561166c17cd4434b43162aca4ff7471 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:18:12.253625 17104 raft_consensus.cc:729] T 00000000000000000000000000000000 P a561166c17cd4434b43162aca4ff7471 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a561166c17cd4434b43162aca4ff7471, State: Initialized, Role: FOLLOWER
I20260807 18:18:12.254307 17104 consensus_queue.cc:260] T 00000000000000000000000000000000 P a561166c17cd4434b43162aca4ff7471 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a561166c17cd4434b43162aca4ff7471" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43783 } }
I20260807 18:18:12.254655 17104 raft_consensus.cc:388] T 00000000000000000000000000000000 P a561166c17cd4434b43162aca4ff7471 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:18:12.254865 17104 raft_consensus.cc:482] T 00000000000000000000000000000000 P a561166c17cd4434b43162aca4ff7471 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:18:12.255069 17104 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a561166c17cd4434b43162aca4ff7471 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:18:12.257503 17104 raft_consensus.cc:504] T 00000000000000000000000000000000 P a561166c17cd4434b43162aca4ff7471 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a561166c17cd4434b43162aca4ff7471" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43783 } }
I20260807 18:18:12.258154 17104 leader_election.cc:302] T 00000000000000000000000000000000 P a561166c17cd4434b43162aca4ff7471 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a561166c17cd4434b43162aca4ff7471; no voters:
I20260807 18:18:12.258810 17104 leader_election.cc:288] T 00000000000000000000000000000000 P a561166c17cd4434b43162aca4ff7471 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:18:12.259020 17121 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a561166c17cd4434b43162aca4ff7471 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:18:12.260541 17121 raft_consensus.cc:686] T 00000000000000000000000000000000 P a561166c17cd4434b43162aca4ff7471 [term 1 LEADER]: Becoming Leader. State: Replica: a561166c17cd4434b43162aca4ff7471, State: Running, Role: LEADER
I20260807 18:18:12.261297 17104 sys_catalog.cc:564] T 00000000000000000000000000000000 P a561166c17cd4434b43162aca4ff7471 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:18:12.261468 17121 consensus_queue.cc:237] T 00000000000000000000000000000000 P a561166c17cd4434b43162aca4ff7471 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a561166c17cd4434b43162aca4ff7471" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43783 } }
I20260807 18:18:12.265108 17124 sys_catalog.cc:455] T 00000000000000000000000000000000 P a561166c17cd4434b43162aca4ff7471 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a561166c17cd4434b43162aca4ff7471. Latest consensus state: current_term: 1 leader_uuid: "a561166c17cd4434b43162aca4ff7471" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a561166c17cd4434b43162aca4ff7471" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43783 } } }
I20260807 18:18:12.265681 17124 sys_catalog.cc:458] T 00000000000000000000000000000000 P a561166c17cd4434b43162aca4ff7471 [sys.catalog]: This master's current role is: LEADER
I20260807 18:18:12.266326 17122 sys_catalog.cc:455] T 00000000000000000000000000000000 P a561166c17cd4434b43162aca4ff7471 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a561166c17cd4434b43162aca4ff7471" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a561166c17cd4434b43162aca4ff7471" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43783 } } }
I20260807 18:18:12.266690 17122 sys_catalog.cc:458] T 00000000000000000000000000000000 P a561166c17cd4434b43162aca4ff7471 [sys.catalog]: This master's current role is: LEADER
I20260807 18:18:12.268872 17138 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:18:12.274540 17138 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:18:12.277473 17145 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260807 18:18:12.291419 17138 catalog_manager.cc:1347] Generated new cluster ID: 1a900569ed4e450e82685085779818c4
I20260807 18:18:12.291565 17138 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:18:12.307171 17138 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:18:12.307862 17138 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:18:12.324260 17138 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a561166c17cd4434b43162aca4ff7471: Generated new TSK 0
I20260807 18:18:12.325191 17138 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:18:12.336740 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 17013
W20260807 18:18:12.338039 17094 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.125:60477
W20260807 18:18:12.338831 19048 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 17013: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 18:18:12.338996 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 17013
2026-08-07T18:18:12Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (2328 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786126328808105-19048-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:18:12 dist-test-slave-gkw7 krb5kdc[17149](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:18:12 dist-test-slave-gkw7 krb5kdc[17149](info): set up 2 sockets
Aug 07 18:18:12 dist-test-slave-gkw7 krb5kdc[17149](info): commencing operation
krb5kdc: starting...
W20260807 18:18:13.670882 19048 mini_kdc.cc:121] Time spent starting KDC: real 1.286s user 0.000s sys 0.046s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786126328808105-19048-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786126328808105-19048-0/krb5kdc/test-user.keytab.
Aug 07 18:18:13 dist-test-slave-gkw7 krb5kdc[17149](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126693, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:18:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:18:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786126328808105-19048-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786126328808105-19048-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 18:18:13.997995 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786126328808105-19048-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786126328808105-19048-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786126328808105-19048-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34541
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786126328808105-19048-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34541
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786126328808105-19048-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786126328808105-19048-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786126328808105-19048-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:18:14.354611 17168 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:18:14.354971 17168 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:18:14.355159 17168 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:18:14.362709 17168 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:18:14.362879 17168 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:18:14.362980 17168 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:18:14.363090 17168 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:18:14.363166 17168 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:18:14.373291 17168 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786126328808105-19048-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786126328808105-19048-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34541
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786126328808105-19048-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34541
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786126328808105-19048-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:18:14.374755 17168 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:18:14.376158 17168 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:18:14.388805 17174 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:18:14.388937 17177 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:18:14.388957 17175 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:18:14.390092 17168 server_base.cc:1034] running on GCE node
I20260807 18:18:14.391004 17168 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:18:14.392517 17168 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:18:14.393734 17168 hybrid_clock.cc:648] HybridClock initialized: now 1786126694393693 us; error 31 us; skew 500 ppm
I20260807 18:18:14.394110 17168 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:18:14 dist-test-slave-gkw7 krb5kdc[17149](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126694, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:18:14.397445 17168 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 18:18:14.399147 17168 webserver.cc:460] Webserver started at http://127.0.0.1:37937/ using document root <none> and password file <none>
I20260807 18:18:14.399873 17168 fs_manager.cc:359] Metadata directory not provided
I20260807 18:18:14.400025 17168 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:18:14.400363 17168 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:18:14.403607 17168 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786126328808105-19048-0/minicluster-data/master-0/data/instance:
uuid: "71d1219b049144e6bc312eb30e6c85c8"
format_stamp: "Formatted at 2026-08-07 18:18:14 on dist-test-slave-gkw7"
server_key: "e475bfba5432f7f385dc89bea3bc4670"
server_key_iv: "f391b06879727e4efc981798cc410842"
server_key_version: "encryptionkey@0"
I20260807 18:18:14.404430 17168 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786126328808105-19048-0/minicluster-data/master-0/wal/instance:
uuid: "71d1219b049144e6bc312eb30e6c85c8"
format_stamp: "Formatted at 2026-08-07 18:18:14 on dist-test-slave-gkw7"
server_key: "e475bfba5432f7f385dc89bea3bc4670"
server_key_iv: "f391b06879727e4efc981798cc410842"
server_key_version: "encryptionkey@0"
I20260807 18:18:14.409968 17168 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260807 18:18:14.413743 17186 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:18:14.415169 17168 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 18:18:14.415380 17168 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786126328808105-19048-0/minicluster-data/master-0/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786126328808105-19048-0/minicluster-data/master-0/wal
uuid: "71d1219b049144e6bc312eb30e6c85c8"
format_stamp: "Formatted at 2026-08-07 18:18:14 on dist-test-slave-gkw7"
server_key: "e475bfba5432f7f385dc89bea3bc4670"
server_key_iv: "f391b06879727e4efc981798cc410842"
server_key_version: "encryptionkey@0"
I20260807 18:18:14.415594 17168 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786126328808105-19048-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786126328808105-19048-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786126328808105-19048-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:18:14.439478 17168 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:18:14.444018 17168 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:18:14.444352 17168 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:18:14.464810 17168 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34541
I20260807 18:18:14.464859 17268 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34541 every 8 connection(s)
I20260807 18:18:14.466418 17168 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
I20260807 18:18:14.470638 17269 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:18:14.471652 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 17168
I20260807 18:18:14.471860 19048 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786126328808105-19048-0/minicluster-data/master-0/wal/instance
I20260807 18:18:14.472373 19048 external_mini_cluster.cc:1428] Setting key ce5f95907e18ddd9aff6a39489966c5a
I20260807 18:18:14.487313 17269 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 71d1219b049144e6bc312eb30e6c85c8: Bootstrap starting.
Aug 07 18:18:14 dist-test-slave-gkw7 krb5kdc[17149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126693, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260807 18:18:14.490844 17269 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 71d1219b049144e6bc312eb30e6c85c8: Neither blocks nor log segments found. Creating new log.
I20260807 18:18:14.492233 17269 log.cc:824] T 00000000000000000000000000000000 P 71d1219b049144e6bc312eb30e6c85c8: Log is configured to *not* fsync() on all Append() calls
I20260807 18:18:14.495657 17269 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 71d1219b049144e6bc312eb30e6c85c8: No bootstrap required, opened a new log
I20260807 18:18:14.500473 17272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:14.474499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.125:41439 (local address 127.0.0.1:34541)
0807 18:18:14.475417 (+ 918us) server_negotiation.cc:206] Beginning negotiation
0807 18:18:14.475433 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 18:18:14.475488 (+ 55us) server_negotiation.cc:407] Connection header received
0807 18:18:14.478277 (+ 2789us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:18:14.478309 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:18:14.478910 (+ 601us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:18:14.479454 (+ 544us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:18:14.480840 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:14.483261 (+ 2421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:14.484584 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:14.485842 (+ 1258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:14.486060 (+ 218us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:18:14.491198 (+ 5138us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:18:14.491208 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:18:14.491228 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:18:14.491280 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:18:14.495002 (+ 3722us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:18:14.495958 (+ 956us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:18:14.495967 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:18:14.495976 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:18:14.496242 (+ 266us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:18:14.496777 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:18:14.496782 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:18:14.496786 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:18:14.497583 (+ 797us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:18:14.497984 (+ 401us) server_negotiation.cc:1035] Waiting for connection context
0807 18:18:14.498502 (+ 518us) server_negotiation.cc:299] Negotiation successful
0807 18:18:14.498910 (+ 408us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":632,"thread_start_us":350,"threads_started":1}
I20260807 18:18:14.504746 19048 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:18:14.506657 17269 raft_consensus.cc:348] T 00000000000000000000000000000000 P 71d1219b049144e6bc312eb30e6c85c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71d1219b049144e6bc312eb30e6c85c8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34541 } }
I20260807 18:18:14.507130 17269 raft_consensus.cc:374] T 00000000000000000000000000000000 P 71d1219b049144e6bc312eb30e6c85c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:18:14.507247 17269 raft_consensus.cc:729] T 00000000000000000000000000000000 P 71d1219b049144e6bc312eb30e6c85c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71d1219b049144e6bc312eb30e6c85c8, State: Initialized, Role: FOLLOWER
I20260807 18:18:14.507984 17269 consensus_queue.cc:260] T 00000000000000000000000000000000 P 71d1219b049144e6bc312eb30e6c85c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71d1219b049144e6bc312eb30e6c85c8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34541 } }
I20260807 18:18:14.508324 17269 raft_consensus.cc:388] T 00000000000000000000000000000000 P 71d1219b049144e6bc312eb30e6c85c8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:18:14.508500 17269 raft_consensus.cc:482] T 00000000000000000000000000000000 P 71d1219b049144e6bc312eb30e6c85c8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:18:14.508744 17269 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 71d1219b049144e6bc312eb30e6c85c8 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:18:14.512143 17269 raft_consensus.cc:504] T 00000000000000000000000000000000 P 71d1219b049144e6bc312eb30e6c85c8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71d1219b049144e6bc312eb30e6c85c8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34541 } }
I20260807 18:18:14.512707 17269 leader_election.cc:302] T 00000000000000000000000000000000 P 71d1219b049144e6bc312eb30e6c85c8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 71d1219b049144e6bc312eb30e6c85c8; no voters:
I20260807 18:18:14.513378 17269 leader_election.cc:288] T 00000000000000000000000000000000 P 71d1219b049144e6bc312eb30e6c85c8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:18:14.513522 17283 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 71d1219b049144e6bc312eb30e6c85c8 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:18:14.515090 17283 raft_consensus.cc:686] T 00000000000000000000000000000000 P 71d1219b049144e6bc312eb30e6c85c8 [term 1 LEADER]: Becoming Leader. State: Replica: 71d1219b049144e6bc312eb30e6c85c8, State: Running, Role: LEADER
I20260807 18:18:14.515646 17283 consensus_queue.cc:237] T 00000000000000000000000000000000 P 71d1219b049144e6bc312eb30e6c85c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71d1219b049144e6bc312eb30e6c85c8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34541 } }
I20260807 18:18:14.515962 17269 sys_catalog.cc:564] T 00000000000000000000000000000000 P 71d1219b049144e6bc312eb30e6c85c8 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:18:14.518366 17285 sys_catalog.cc:455] T 00000000000000000000000000000000 P 71d1219b049144e6bc312eb30e6c85c8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 71d1219b049144e6bc312eb30e6c85c8. Latest consensus state: current_term: 1 leader_uuid: "71d1219b049144e6bc312eb30e6c85c8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71d1219b049144e6bc312eb30e6c85c8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34541 } } }
I20260807 18:18:14.518761 17285 sys_catalog.cc:458] T 00000000000000000000000000000000 P 71d1219b049144e6bc312eb30e6c85c8 [sys.catalog]: This master's current role is: LEADER
I20260807 18:18:14.519482 17284 sys_catalog.cc:455] T 00000000000000000000000000000000 P 71d1219b049144e6bc312eb30e6c85c8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "71d1219b049144e6bc312eb30e6c85c8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71d1219b049144e6bc312eb30e6c85c8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34541 } } }
I20260807 18:18:14.519860 17284 sys_catalog.cc:458] T 00000000000000000000000000000000 P 71d1219b049144e6bc312eb30e6c85c8 [sys.catalog]: This master's current role is: LEADER
I20260807 18:18:14.529855 17291 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:18:14.532186 17272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:14.510666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.125:35279 (local address 127.0.0.1:34541)
0807 18:18:14.510865 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0807 18:18:14.510871 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:18:14.511246 (+ 375us) server_negotiation.cc:407] Connection header received
0807 18:18:14.511383 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:18:14.511389 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:18:14.511474 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:18:14.511588 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:18:14.513323 (+ 1735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:14.514797 (+ 1474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:14.516903 (+ 2106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:14.517912 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:14.519034 (+ 1122us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:18:14.522955 (+ 3921us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:18:14.522966 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:18:14.522975 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:18:14.523032 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:18:14.528049 (+ 5017us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:18:14.529263 (+ 1214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:18:14.529270 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:18:14.529276 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:18:14.529464 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:18:14.530198 (+ 734us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:18:14.530203 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:18:14.530206 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:18:14.530660 (+ 454us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:18:14.530956 (+ 296us) server_negotiation.cc:1035] Waiting for connection context
0807 18:18:14.531608 (+ 652us) server_negotiation.cc:299] Negotiation successful
0807 18:18:14.531798 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260807 18:18:14.536854 17291 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:18:14.547173 17291 catalog_manager.cc:1347] Generated new cluster ID: 3a92fa98ef734b8abcfe9ce3b4147827
I20260807 18:18:14.547358 17291 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:18:14.568008 17291 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:18:14.569489 17291 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:18:14.585230 17291 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 71d1219b049144e6bc312eb30e6c85c8: Generated new TSK 0
I20260807 18:18:14.586369 17291 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:18:14.604425 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 17168
I20260807 18:18:14.606034 17259 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:18:14.703016 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 17168
W20260807 18:18:14.727249 17147 connection.cc:535] client connection to 127.0.0.1:34541 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34541): Connection reset by peer (error 104)
2026-08-07T18:18:14Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (2361 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786126328808105-19048-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:18:14 dist-test-slave-gkw7 krb5kdc[17310](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:18:14 dist-test-slave-gkw7 krb5kdc[17310](info): set up 2 sockets
Aug 07 18:18:14 dist-test-slave-gkw7 krb5kdc[17310](info): commencing operation
krb5kdc: starting...
W20260807 18:18:16.044366 19048 mini_kdc.cc:121] Time spent starting KDC: real 1.298s user 0.002s sys 0.045s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786126328808105-19048-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786126328808105-19048-0/krb5kdc/test-user.keytab.
Aug 07 18:18:16 dist-test-slave-gkw7 krb5kdc[17310](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126696, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:18:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:18:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786126328808105-19048-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786126328808105-19048-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 18:18:16.394517 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786126328808105-19048-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786126328808105-19048-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786126328808105-19048-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39181
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786126328808105-19048-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33355
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39181
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786126328808105-19048-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786126328808105-19048-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786126328808105-19048-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:18:16.772583 17328 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:18:16.772904 17328 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:18:16.773116 17328 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:18:16.780766 17328 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:18:16.780947 17328 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:18:16.781051 17328 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:18:16.781150 17328 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:18:16.781251 17328 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:18:16.791669 17328 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33355
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786126328808105-19048-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786126328808105-19048-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39181
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786126328808105-19048-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39181
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786126328808105-19048-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:18:16.793078 17328 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:18:16.794553 17328 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:18:16.808146 17334 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:18:16.808241 17338 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:18:16.808215 17336 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:18:16.809211 17328 server_base.cc:1034] running on GCE node
I20260807 18:18:16.810212 17328 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:18:16.811913 17328 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:18:16.813174 17328 hybrid_clock.cc:648] HybridClock initialized: now 1786126696813123 us; error 32 us; skew 500 ppm
I20260807 18:18:16.813694 17328 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:18:16 dist-test-slave-gkw7 krb5kdc[17310](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126696, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:18:16.817328 17328 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 18:18:16.819151 17328 webserver.cc:460] Webserver started at http://127.0.0.1:42957/ using document root <none> and password file <none>
I20260807 18:18:16.819927 17328 fs_manager.cc:359] Metadata directory not provided
I20260807 18:18:16.820096 17328 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:18:16.820452 17328 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:18:16.823990 17328 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786126328808105-19048-0/minicluster-data/master-0/data/instance:
uuid: "f246142008674ee9a1cbec8ac93e049e"
format_stamp: "Formatted at 2026-08-07 18:18:16 on dist-test-slave-gkw7"
server_key: "fec8f5637ba79dc3cb00c3de20085983"
server_key_iv: "a1444e9f5f392eb082a0bedafdded38b"
server_key_version: "encryptionkey@0"
I20260807 18:18:16.824883 17328 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786126328808105-19048-0/minicluster-data/master-0/wal/instance:
uuid: "f246142008674ee9a1cbec8ac93e049e"
format_stamp: "Formatted at 2026-08-07 18:18:16 on dist-test-slave-gkw7"
server_key: "fec8f5637ba79dc3cb00c3de20085983"
server_key_iv: "a1444e9f5f392eb082a0bedafdded38b"
server_key_version: "encryptionkey@0"
I20260807 18:18:16.830915 17328 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.001s sys 0.007s
I20260807 18:18:16.835069 17352 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:18:16.836762 17328 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260807 18:18:16.836933 17328 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786126328808105-19048-0/minicluster-data/master-0/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786126328808105-19048-0/minicluster-data/master-0/wal
uuid: "f246142008674ee9a1cbec8ac93e049e"
format_stamp: "Formatted at 2026-08-07 18:18:16 on dist-test-slave-gkw7"
server_key: "fec8f5637ba79dc3cb00c3de20085983"
server_key_iv: "a1444e9f5f392eb082a0bedafdded38b"
server_key_version: "encryptionkey@0"
I20260807 18:18:16.837222 17328 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786126328808105-19048-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786126328808105-19048-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786126328808105-19048-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:18:16.873365 17328 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:18:16.877986 17328 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:18:16.878319 17328 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:18:16.898335 17328 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39181
I20260807 18:18:16.898438 17425 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39181 every 8 connection(s)
I20260807 18:18:16.899883 17328 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
I20260807 18:18:16.904026 17426 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:18:16.908962 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 17328
I20260807 18:18:16.909174 19048 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786126328808105-19048-0/minicluster-data/master-0/wal/instance
I20260807 18:18:16.909770 19048 external_mini_cluster.cc:1428] Setting key d4e2df49518db7e9e12ae9f40a2273a9
I20260807 18:18:16.915967 17426 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f246142008674ee9a1cbec8ac93e049e: Bootstrap starting.
I20260807 18:18:16.919261 17426 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f246142008674ee9a1cbec8ac93e049e: Neither blocks nor log segments found. Creating new log.
I20260807 18:18:16.920728 17426 log.cc:824] T 00000000000000000000000000000000 P f246142008674ee9a1cbec8ac93e049e: Log is configured to *not* fsync() on all Append() calls
I20260807 18:18:16.923805 17429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:16.911772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.125:60781 (local address 127.0.0.1:39181)
0807 18:18:16.912579 (+ 807us) server_negotiation.cc:206] Beginning negotiation
0807 18:18:16.912591 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 18:18:16.912633 (+ 42us) server_negotiation.cc:407] Connection header received
0807 18:18:16.914613 (+ 1980us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:18:16.914638 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:18:16.915045 (+ 407us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:18:16.915436 (+ 391us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:18:16.915989 (+ 553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:16.917352 (+ 1363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:16.918761 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:16.919556 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:16.919771 (+ 215us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:18:16.921053 (+ 1282us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:18:16.921061 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:18:16.921074 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:18:16.921924 (+ 850us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:18:16.922030 (+ 106us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:18:16.922069 (+ 39us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:18:16.922254 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0807 18:18:16.922593 (+ 339us) server_negotiation.cc:299] Negotiation successful
0807 18:18:16.922801 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":581,"thread_start_us":364,"threads_started":1}
I20260807 18:18:16.924134 17426 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f246142008674ee9a1cbec8ac93e049e: No bootstrap required, opened a new log
I20260807 18:18:16.927742 19048 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:18:16.934152 17426 raft_consensus.cc:348] T 00000000000000000000000000000000 P f246142008674ee9a1cbec8ac93e049e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f246142008674ee9a1cbec8ac93e049e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39181 } }
I20260807 18:18:16.934729 17426 raft_consensus.cc:374] T 00000000000000000000000000000000 P f246142008674ee9a1cbec8ac93e049e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:18:16.934891 17426 raft_consensus.cc:729] T 00000000000000000000000000000000 P f246142008674ee9a1cbec8ac93e049e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f246142008674ee9a1cbec8ac93e049e, State: Initialized, Role: FOLLOWER
I20260807 18:18:16.935503 17426 consensus_queue.cc:260] T 00000000000000000000000000000000 P f246142008674ee9a1cbec8ac93e049e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f246142008674ee9a1cbec8ac93e049e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39181 } }
I20260807 18:18:16.935765 17426 raft_consensus.cc:388] T 00000000000000000000000000000000 P f246142008674ee9a1cbec8ac93e049e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:18:16.935935 17426 raft_consensus.cc:482] T 00000000000000000000000000000000 P f246142008674ee9a1cbec8ac93e049e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:18:16.936129 17426 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f246142008674ee9a1cbec8ac93e049e [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:18:16.938510 17426 raft_consensus.cc:504] T 00000000000000000000000000000000 P f246142008674ee9a1cbec8ac93e049e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f246142008674ee9a1cbec8ac93e049e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39181 } }
I20260807 18:18:16.939124 17426 leader_election.cc:302] T 00000000000000000000000000000000 P f246142008674ee9a1cbec8ac93e049e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f246142008674ee9a1cbec8ac93e049e; no voters:
I20260807 18:18:16.939845 17426 leader_election.cc:288] T 00000000000000000000000000000000 P f246142008674ee9a1cbec8ac93e049e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:18:16.940044 17442 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f246142008674ee9a1cbec8ac93e049e [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:18:16.940897 17429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:16.933058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.125:53619 (local address 127.0.0.1:39181)
0807 18:18:16.933298 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0807 18:18:16.933309 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 18:18:16.933828 (+ 519us) server_negotiation.cc:407] Connection header received
0807 18:18:16.934018 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:18:16.934022 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:18:16.934103 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:18:16.934199 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:18:16.934851 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:16.936017 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:16.937480 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:16.938073 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:16.938222 (+ 149us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:18:16.939488 (+ 1266us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:18:16.939496 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:18:16.939504 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:18:16.939532 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:18:16.939609 (+ 77us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:18:16.939643 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:18:16.939783 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0807 18:18:16.940282 (+ 499us) server_negotiation.cc:299] Negotiation successful
0807 18:18:16.940368 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260807 18:18:16.940948 17442 raft_consensus.cc:686] T 00000000000000000000000000000000 P f246142008674ee9a1cbec8ac93e049e [term 1 LEADER]: Becoming Leader. State: Replica: f246142008674ee9a1cbec8ac93e049e, State: Running, Role: LEADER
I20260807 18:18:16.942395 17442 consensus_queue.cc:237] T 00000000000000000000000000000000 P f246142008674ee9a1cbec8ac93e049e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f246142008674ee9a1cbec8ac93e049e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39181 } }
I20260807 18:18:16.945474 17444 sys_catalog.cc:455] T 00000000000000000000000000000000 P f246142008674ee9a1cbec8ac93e049e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f246142008674ee9a1cbec8ac93e049e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f246142008674ee9a1cbec8ac93e049e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39181 } } }
I20260807 18:18:16.945940 17444 sys_catalog.cc:458] T 00000000000000000000000000000000 P f246142008674ee9a1cbec8ac93e049e [sys.catalog]: This master's current role is: LEADER
I20260807 18:18:16.946520 17442 sys_catalog.cc:455] T 00000000000000000000000000000000 P f246142008674ee9a1cbec8ac93e049e [sys.catalog]: SysCatalogTable state changed. Reason: New leader f246142008674ee9a1cbec8ac93e049e. Latest consensus state: current_term: 1 leader_uuid: "f246142008674ee9a1cbec8ac93e049e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f246142008674ee9a1cbec8ac93e049e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39181 } } }
I20260807 18:18:16.946707 17426 sys_catalog.cc:564] T 00000000000000000000000000000000 P f246142008674ee9a1cbec8ac93e049e [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:18:16.946740 17442 sys_catalog.cc:458] T 00000000000000000000000000000000 P f246142008674ee9a1cbec8ac93e049e [sys.catalog]: This master's current role is: LEADER
I20260807 18:18:16.947716 17446 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:18:16.954525 17446 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:18:16.964490 17446 catalog_manager.cc:1347] Generated new cluster ID: b55202c866b747a1b3d654cd65e448cf
I20260807 18:18:16.964632 17446 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:18:16.978814 17446 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:18:16.980903 17446 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:18:16.991147 17446 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f246142008674ee9a1cbec8ac93e049e: Generated new TSK 0
I20260807 18:18:16.992419 17446 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:18:17.006333 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 17328
W20260807 18:18:17.008116 17418 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.125:60781
W20260807 18:18:17.008963 19048 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 17328: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 18:18:17.009099 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 17328
W20260807 18:18:17.033978 17308 connection.cc:535] client connection to 127.0.0.1:39181 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39181): Connection reset by peer (error 104)
2026-08-07T18:18:17Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (2306 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786126328808105-19048-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:18:17 dist-test-slave-gkw7 krb5kdc[17466](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:18:17 dist-test-slave-gkw7 krb5kdc[17466](info): set up 2 sockets
Aug 07 18:18:17 dist-test-slave-gkw7 krb5kdc[17466](info): commencing operation
krb5kdc: starting...
W20260807 18:18:18.337414 19048 mini_kdc.cc:121] Time spent starting KDC: real 1.283s user 0.000s sys 0.049s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786126328808105-19048-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786126328808105-19048-0/krb5kdc/test-user.keytab.
Aug 07 18:18:18 dist-test-slave-gkw7 krb5kdc[17466](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126698, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:18:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:18:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786126328808105-19048-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786126328808105-19048-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 18:18:18.679553 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786126328808105-19048-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786126328808105-19048-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786126328808105-19048-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44513
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786126328808105-19048-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39079
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44513
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786126328808105-19048-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786126328808105-19048-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786126328808105-19048-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:18:19.032537 17485 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:18:19.032922 17485 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:18:19.033123 17485 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:18:19.040587 17485 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:18:19.040755 17485 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:18:19.040864 17485 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:18:19.040972 17485 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:18:19.041064 17485 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:18:19.051129 17485 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39079
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786126328808105-19048-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786126328808105-19048-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44513
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786126328808105-19048-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44513
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786126328808105-19048-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:18:19.052551 17485 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:18:19.053963 17485 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:18:19.066491 17492 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:18:19.066437 17493 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:18:19.066717 17496 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:18:19.067436 17485 server_base.cc:1034] running on GCE node
I20260807 18:18:19.068460 17485 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:18:19.070191 17485 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:18:19.071476 17485 hybrid_clock.cc:648] HybridClock initialized: now 1786126699071416 us; error 49 us; skew 500 ppm
I20260807 18:18:19.071943 17485 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:18:19 dist-test-slave-gkw7 krb5kdc[17466](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126699, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:18:19.080312 17485 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 18:18:19.082243 17485 webserver.cc:460] Webserver started at http://127.0.0.1:35253/ using document root <none> and password file <none>
I20260807 18:18:19.083022 17485 fs_manager.cc:359] Metadata directory not provided
I20260807 18:18:19.083201 17485 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:18:19.083551 17485 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:18:19.086930 17485 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786126328808105-19048-0/minicluster-data/master-0/data/instance:
uuid: "5fb6e85f885c4cb8bb1b013723b9dd5b"
format_stamp: "Formatted at 2026-08-07 18:18:19 on dist-test-slave-gkw7"
server_key: "3016684a7ccc5c247c5fbd2907f799fa"
server_key_iv: "6a88c17cd1e8d9d7d2142537cfc6e250"
server_key_version: "encryptionkey@0"
I20260807 18:18:19.087765 17485 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786126328808105-19048-0/minicluster-data/master-0/wal/instance:
uuid: "5fb6e85f885c4cb8bb1b013723b9dd5b"
format_stamp: "Formatted at 2026-08-07 18:18:19 on dist-test-slave-gkw7"
server_key: "3016684a7ccc5c247c5fbd2907f799fa"
server_key_iv: "6a88c17cd1e8d9d7d2142537cfc6e250"
server_key_version: "encryptionkey@0"
I20260807 18:18:19.093727 17485 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260807 18:18:19.097700 17505 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:18:19.099290 17485 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:18:19.099517 17485 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786126328808105-19048-0/minicluster-data/master-0/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786126328808105-19048-0/minicluster-data/master-0/wal
uuid: "5fb6e85f885c4cb8bb1b013723b9dd5b"
format_stamp: "Formatted at 2026-08-07 18:18:19 on dist-test-slave-gkw7"
server_key: "3016684a7ccc5c247c5fbd2907f799fa"
server_key_iv: "6a88c17cd1e8d9d7d2142537cfc6e250"
server_key_version: "encryptionkey@0"
I20260807 18:18:19.099772 17485 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786126328808105-19048-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786126328808105-19048-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786126328808105-19048-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:18:19.106678 17485 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:18:19.111937 17485 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:18:19.112356 17485 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:18:19.133821 17485 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44513
I20260807 18:18:19.133862 17575 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44513 every 8 connection(s)
I20260807 18:18:19.135533 17485 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
I20260807 18:18:19.139816 17576 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:18:19.143356 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 17485
I20260807 18:18:19.143566 19048 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786126328808105-19048-0/minicluster-data/master-0/wal/instance
I20260807 18:18:19.144097 19048 external_mini_cluster.cc:1428] Setting key 1a3c426056e6760e567597032dddb3d0
I20260807 18:18:19.154101 17581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:19.146267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.125:60635 (local address 127.0.0.1:44513)
0807 18:18:19.147139 (+ 872us) server_negotiation.cc:206] Beginning negotiation
0807 18:18:19.147157 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0807 18:18:19.147213 (+ 56us) server_negotiation.cc:407] Connection header received
0807 18:18:19.149746 (+ 2533us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:18:19.149785 (+ 39us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:18:19.150310 (+ 525us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:18:19.150797 (+ 487us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:18:19.151899 (+ 1102us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:18:19.151908 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:18:19.151925 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:18:19.151952 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:18:19.152044 (+ 92us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:18:19.152075 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:18:19.152353 (+ 278us) server_negotiation.cc:1035] Waiting for connection context
0807 18:18:19.152503 (+ 150us) server_negotiation.cc:299] Negotiation successful
0807 18:18:19.152704 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":601,"thread_start_us":341,"threads_started":1}
I20260807 18:18:19.154865 17576 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5fb6e85f885c4cb8bb1b013723b9dd5b: Bootstrap starting.
I20260807 18:18:19.158375 19048 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:18:19.158658 17576 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5fb6e85f885c4cb8bb1b013723b9dd5b: Neither blocks nor log segments found. Creating new log.
I20260807 18:18:19.160055 17576 log.cc:824] T 00000000000000000000000000000000 P 5fb6e85f885c4cb8bb1b013723b9dd5b: Log is configured to *not* fsync() on all Append() calls
I20260807 18:18:19.163794 17576 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5fb6e85f885c4cb8bb1b013723b9dd5b: No bootstrap required, opened a new log
I20260807 18:18:19.167140 17581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:19.163993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.125:49923 (local address 127.0.0.1:44513)
0807 18:18:19.164228 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0807 18:18:19.164248 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0807 18:18:19.164729 (+ 481us) server_negotiation.cc:407] Connection header received
0807 18:18:19.164976 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:18:19.164980 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:18:19.165031 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:18:19.165172 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:18:19.166360 (+ 1188us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:18:19.166364 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:18:19.166368 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:18:19.166384 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:18:19.166428 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:18:19.166444 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:18:19.166543 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0807 18:18:19.166708 (+ 165us) server_negotiation.cc:299] Negotiation successful
0807 18:18:19.166761 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260807 18:18:19.172816 17576 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5fb6e85f885c4cb8bb1b013723b9dd5b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5fb6e85f885c4cb8bb1b013723b9dd5b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44513 } }
I20260807 18:18:19.173264 17576 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5fb6e85f885c4cb8bb1b013723b9dd5b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:18:19.173385 17576 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5fb6e85f885c4cb8bb1b013723b9dd5b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5fb6e85f885c4cb8bb1b013723b9dd5b, State: Initialized, Role: FOLLOWER
I20260807 18:18:19.174059 17576 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5fb6e85f885c4cb8bb1b013723b9dd5b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5fb6e85f885c4cb8bb1b013723b9dd5b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44513 } }
I20260807 18:18:19.174336 17576 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5fb6e85f885c4cb8bb1b013723b9dd5b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:18:19.174499 17576 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5fb6e85f885c4cb8bb1b013723b9dd5b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:18:19.174688 17576 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5fb6e85f885c4cb8bb1b013723b9dd5b [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:18:19.177134 17576 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5fb6e85f885c4cb8bb1b013723b9dd5b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5fb6e85f885c4cb8bb1b013723b9dd5b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44513 } }
I20260807 18:18:19.177760 17576 leader_election.cc:302] T 00000000000000000000000000000000 P 5fb6e85f885c4cb8bb1b013723b9dd5b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5fb6e85f885c4cb8bb1b013723b9dd5b; no voters:
I20260807 18:18:19.178377 17576 leader_election.cc:288] T 00000000000000000000000000000000 P 5fb6e85f885c4cb8bb1b013723b9dd5b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:18:19.178511 17594 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5fb6e85f885c4cb8bb1b013723b9dd5b [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:18:19.179384 17594 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5fb6e85f885c4cb8bb1b013723b9dd5b [term 1 LEADER]: Becoming Leader. State: Replica: 5fb6e85f885c4cb8bb1b013723b9dd5b, State: Running, Role: LEADER
I20260807 18:18:19.180017 17594 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5fb6e85f885c4cb8bb1b013723b9dd5b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5fb6e85f885c4cb8bb1b013723b9dd5b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44513 } }
I20260807 18:18:19.180853 17576 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5fb6e85f885c4cb8bb1b013723b9dd5b [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:18:19.182933 17595 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5fb6e85f885c4cb8bb1b013723b9dd5b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5fb6e85f885c4cb8bb1b013723b9dd5b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5fb6e85f885c4cb8bb1b013723b9dd5b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44513 } } }
I20260807 18:18:19.183362 17595 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5fb6e85f885c4cb8bb1b013723b9dd5b [sys.catalog]: This master's current role is: LEADER
I20260807 18:18:19.183648 17597 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5fb6e85f885c4cb8bb1b013723b9dd5b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5fb6e85f885c4cb8bb1b013723b9dd5b. Latest consensus state: current_term: 1 leader_uuid: "5fb6e85f885c4cb8bb1b013723b9dd5b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5fb6e85f885c4cb8bb1b013723b9dd5b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44513 } } }
I20260807 18:18:19.184029 17597 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5fb6e85f885c4cb8bb1b013723b9dd5b [sys.catalog]: This master's current role is: LEADER
I20260807 18:18:19.190478 17600 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:18:19.196563 17600 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:18:19.206207 17600 catalog_manager.cc:1347] Generated new cluster ID: 59363b05a40f4117b3e4059cbe07cc84
I20260807 18:18:19.206352 17600 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:18:19.237813 17600 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:18:19.238385 17600 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:18:19.258172 17600 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5fb6e85f885c4cb8bb1b013723b9dd5b: Generated new TSK 0
I20260807 18:18:19.259204 17600 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:18:19.280279 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 17485
W20260807 18:18:19.282881 17567 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.125:60635
W20260807 18:18:19.283697 19048 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 17485: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 18:18:19.283874 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 17485
2026-08-07T18:18:19Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (2276 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786126328808105-19048-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:18:19 dist-test-slave-gkw7 krb5kdc[17623](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:18:19 dist-test-slave-gkw7 krb5kdc[17623](info): set up 2 sockets
Aug 07 18:18:19 dist-test-slave-gkw7 krb5kdc[17623](info): commencing operation
krb5kdc: starting...
W20260807 18:18:20.644551 19048 mini_kdc.cc:121] Time spent starting KDC: real 1.313s user 0.002s sys 0.043s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786126328808105-19048-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786126328808105-19048-0/krb5kdc/test-user.keytab.
Aug 07 18:18:20 dist-test-slave-gkw7 krb5kdc[17623](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126700, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:18:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:18:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.154.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.154.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.154.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.62.keytab.
Entry for principal kudu/127.18.154.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.62.keytab.
I20260807 18:18:20.985749 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786126328808105-19048-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786126328808105-19048-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786126328808105-19048-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.154.62:44315
--webserver_interface=127.18.154.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.62.keytab
--principal=kudu/127.18.154.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.154.20:38855
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.154.62:44315
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786126328808105-19048-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786126328808105-19048-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786126328808105-19048-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:18:21.352841 17637 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:18:21.353160 17637 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:18:21.353257 17637 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:18:21.362195 17637 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:18:21.362318 17637 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:18:21.362366 17637 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:18:21.362421 17637 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:18:21.362468 17637 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:18:21.372824 17637 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.154.20:38855
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786126328808105-19048-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786126328808105-19048-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.154.62:44315
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.62.keytab
--principal=kudu/127.18.154.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.154.62:44315
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.154.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786126328808105-19048-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:18:21.374320 17637 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:18:21.375663 17637 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:18:21.388073 17648 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:18:21.388144 17644 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:18:21.388170 17645 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:18:21.391198 17637 server_base.cc:1034] running on GCE node
I20260807 18:18:21.391942 17637 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:18:21.393349 17637 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:18:21.394556 17637 hybrid_clock.cc:648] HybridClock initialized: now 1786126701394489 us; error 65 us; skew 500 ppm
I20260807 18:18:21.394922 17637 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:18:21 dist-test-slave-gkw7 krb5kdc[17623](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126701, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:18:21.398290 17637 init.cc:377] Logged in from keytab as kudu/127.18.154.62@KRBTEST.COM (short username kudu)
I20260807 18:18:21.399963 17637 webserver.cc:460] Webserver started at http://127.18.154.62:41279/ using document root <none> and password file <none>
I20260807 18:18:21.400724 17637 fs_manager.cc:359] Metadata directory not provided
I20260807 18:18:21.400876 17637 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:18:21.401211 17637 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:18:21.404378 17637 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786126328808105-19048-0/minicluster-data/master-0/data/instance:
uuid: "380f6db554134ec7abf1bbbca4bbc393"
format_stamp: "Formatted at 2026-08-07 18:18:21 on dist-test-slave-gkw7"
server_key: "d5dbada20fdabf5cf63fd97f65c74d6d"
server_key_iv: "5a23e6d807a449566531e591d401557f"
server_key_version: "encryptionkey@0"
I20260807 18:18:21.405189 17637 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786126328808105-19048-0/minicluster-data/master-0/wal/instance:
uuid: "380f6db554134ec7abf1bbbca4bbc393"
format_stamp: "Formatted at 2026-08-07 18:18:21 on dist-test-slave-gkw7"
server_key: "d5dbada20fdabf5cf63fd97f65c74d6d"
server_key_iv: "5a23e6d807a449566531e591d401557f"
server_key_version: "encryptionkey@0"
I20260807 18:18:21.410833 17637 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260807 18:18:21.414947 17660 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:18:21.416381 17637 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.002s
I20260807 18:18:21.416597 17637 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786126328808105-19048-0/minicluster-data/master-0/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786126328808105-19048-0/minicluster-data/master-0/wal
uuid: "380f6db554134ec7abf1bbbca4bbc393"
format_stamp: "Formatted at 2026-08-07 18:18:21 on dist-test-slave-gkw7"
server_key: "d5dbada20fdabf5cf63fd97f65c74d6d"
server_key_iv: "5a23e6d807a449566531e591d401557f"
server_key_version: "encryptionkey@0"
I20260807 18:18:21.416831 17637 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786126328808105-19048-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786126328808105-19048-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786126328808105-19048-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:18:21.437647 17637 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:18:21.442345 17637 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:18:21.442659 17637 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:18:21.462287 17637 rpc_server.cc:307] RPC server started. Bound to: 127.18.154.62:44315
I20260807 18:18:21.462320 17728 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.154.62:44315 every 8 connection(s)
I20260807 18:18:21.463833 17637 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
I20260807 18:18:21.467792 17729 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:18:21.470152 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 17637
I20260807 18:18:21.470347 19048 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786126328808105-19048-0/minicluster-data/master-0/wal/instance
I20260807 18:18:21.470829 19048 external_mini_cluster.cc:1428] Setting key fff1878825f09576dc15f3554fed6747
I20260807 18:18:21.480792 17729 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 380f6db554134ec7abf1bbbca4bbc393: Bootstrap starting.
I20260807 18:18:21.485137 17729 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 380f6db554134ec7abf1bbbca4bbc393: Neither blocks nor log segments found. Creating new log.
I20260807 18:18:21.486727 17729 log.cc:824] T 00000000000000000000000000000000 P 380f6db554134ec7abf1bbbca4bbc393: Log is configured to *not* fsync() on all Append() calls
Aug 07 18:18:21 dist-test-slave-gkw7 krb5kdc[17623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126700, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.154.62@KRBTEST.COM
I20260807 18:18:21.490191 17729 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 380f6db554134ec7abf1bbbca4bbc393: No bootstrap required, opened a new log
I20260807 18:18:21.497651 17734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:21.473002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42351 (local address 127.18.154.62:44315)
0807 18:18:21.474063 (+ 1061us) server_negotiation.cc:206] Beginning negotiation
0807 18:18:21.474084 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0807 18:18:21.474130 (+ 46us) server_negotiation.cc:407] Connection header received
0807 18:18:21.475997 (+ 1867us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:18:21.476023 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:18:21.476486 (+ 463us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:18:21.476898 (+ 412us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:18:21.478840 (+ 1942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:21.480214 (+ 1374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:21.482148 (+ 1934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:21.482867 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:21.483003 (+ 136us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:18:21.488710 (+ 5707us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:18:21.488718 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:18:21.488736 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:18:21.488773 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:18:21.492014 (+ 3241us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:18:21.493039 (+ 1025us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:18:21.493050 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:18:21.493066 (+ 16us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:18:21.493313 (+ 247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:18:21.494009 (+ 696us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:18:21.494015 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:18:21.494019 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:18:21.494853 (+ 834us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:18:21.495181 (+ 328us) server_negotiation.cc:1035] Waiting for connection context
0807 18:18:21.495670 (+ 489us) server_negotiation.cc:299] Negotiation successful
0807 18:18:21.496094 (+ 424us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":826,"thread_start_us":509,"threads_started":1}
I20260807 18:18:21.500367 17729 raft_consensus.cc:348] T 00000000000000000000000000000000 P 380f6db554134ec7abf1bbbca4bbc393 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "380f6db554134ec7abf1bbbca4bbc393" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 44315 } }
I20260807 18:18:21.500799 17729 raft_consensus.cc:374] T 00000000000000000000000000000000 P 380f6db554134ec7abf1bbbca4bbc393 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:18:21.500936 17729 raft_consensus.cc:729] T 00000000000000000000000000000000 P 380f6db554134ec7abf1bbbca4bbc393 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 380f6db554134ec7abf1bbbca4bbc393, State: Initialized, Role: FOLLOWER
I20260807 18:18:21.501679 17729 consensus_queue.cc:260] T 00000000000000000000000000000000 P 380f6db554134ec7abf1bbbca4bbc393 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "380f6db554134ec7abf1bbbca4bbc393" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 44315 } }
I20260807 18:18:21.502033 19048 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:18:21.502025 17729 raft_consensus.cc:388] T 00000000000000000000000000000000 P 380f6db554134ec7abf1bbbca4bbc393 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:18:21.502193 17729 raft_consensus.cc:482] T 00000000000000000000000000000000 P 380f6db554134ec7abf1bbbca4bbc393 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:18:21.502389 17729 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 380f6db554134ec7abf1bbbca4bbc393 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:18:21.504662 17729 raft_consensus.cc:504] T 00000000000000000000000000000000 P 380f6db554134ec7abf1bbbca4bbc393 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "380f6db554134ec7abf1bbbca4bbc393" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 44315 } }
I20260807 18:18:21.505343 17729 leader_election.cc:302] T 00000000000000000000000000000000 P 380f6db554134ec7abf1bbbca4bbc393 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 380f6db554134ec7abf1bbbca4bbc393; no voters:
I20260807 18:18:21.506065 17729 leader_election.cc:288] T 00000000000000000000000000000000 P 380f6db554134ec7abf1bbbca4bbc393 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:18:21.506176 17743 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 380f6db554134ec7abf1bbbca4bbc393 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:18:21.509258 17729 sys_catalog.cc:564] T 00000000000000000000000000000000 P 380f6db554134ec7abf1bbbca4bbc393 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:18:21.509836 17743 raft_consensus.cc:686] T 00000000000000000000000000000000 P 380f6db554134ec7abf1bbbca4bbc393 [term 1 LEADER]: Becoming Leader. State: Replica: 380f6db554134ec7abf1bbbca4bbc393, State: Running, Role: LEADER
I20260807 18:18:21.510533 17743 consensus_queue.cc:237] T 00000000000000000000000000000000 P 380f6db554134ec7abf1bbbca4bbc393 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "380f6db554134ec7abf1bbbca4bbc393" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 44315 } }
I20260807 18:18:21.514608 17746 sys_catalog.cc:455] T 00000000000000000000000000000000 P 380f6db554134ec7abf1bbbca4bbc393 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 380f6db554134ec7abf1bbbca4bbc393. Latest consensus state: current_term: 1 leader_uuid: "380f6db554134ec7abf1bbbca4bbc393" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "380f6db554134ec7abf1bbbca4bbc393" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 44315 } } }
I20260807 18:18:21.515111 17746 sys_catalog.cc:458] T 00000000000000000000000000000000 P 380f6db554134ec7abf1bbbca4bbc393 [sys.catalog]: This master's current role is: LEADER
I20260807 18:18:21.518776 17747 sys_catalog.cc:455] T 00000000000000000000000000000000 P 380f6db554134ec7abf1bbbca4bbc393 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "380f6db554134ec7abf1bbbca4bbc393" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "380f6db554134ec7abf1bbbca4bbc393" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 44315 } } }
I20260807 18:18:21.519393 17747 sys_catalog.cc:458] T 00000000000000000000000000000000 P 380f6db554134ec7abf1bbbca4bbc393 [sys.catalog]: This master's current role is: LEADER
I20260807 18:18:21.525975 17754 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:18:21.526876 17734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:21.506614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36857 (local address 127.18.154.62:44315)
0807 18:18:21.506803 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0807 18:18:21.506812 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:18:21.510721 (+ 3909us) server_negotiation.cc:407] Connection header received
0807 18:18:21.510978 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:18:21.510983 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:18:21.511059 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:18:21.511186 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:18:21.513302 (+ 2116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:21.514362 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:21.516128 (+ 1766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:21.516651 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:21.516798 (+ 147us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:18:21.520448 (+ 3650us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:18:21.520455 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:18:21.520461 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:18:21.520499 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:18:21.523771 (+ 3272us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:18:21.524532 (+ 761us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:18:21.524537 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:18:21.524541 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:18:21.524674 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:18:21.525132 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:18:21.525137 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:18:21.525140 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:18:21.525479 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:18:21.525825 (+ 346us) server_negotiation.cc:1035] Waiting for connection context
0807 18:18:21.526310 (+ 485us) server_negotiation.cc:299] Negotiation successful
0807 18:18:21.526500 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 18:18:21.532426 17754 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:18:21.542052 17754 catalog_manager.cc:1347] Generated new cluster ID: e3a19a650468435c9a9767848232aab8
I20260807 18:18:21.542235 17754 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:18:21.554883 17754 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:18:21.556133 17754 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:18:21.570241 17754 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 380f6db554134ec7abf1bbbca4bbc393: Generated new TSK 0
I20260807 18:18:21.571166 17754 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:18:21.587399 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 17637
I20260807 18:18:21.588640 17723 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:18:21.685197 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 17637
W20260807 18:18:21.710876 17621 connection.cc:535] client connection to 127.18.154.62:44315 recv error: Network error: failed to read from TLS socket (remote: 127.18.154.62:44315): Connection reset by peer (error 104)
2026-08-07T18:18:21Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (2399 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786126328808105-19048-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:18:21 dist-test-slave-gkw7 krb5kdc[17773](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:18:21 dist-test-slave-gkw7 krb5kdc[17773](info): set up 2 sockets
Aug 07 18:18:21 dist-test-slave-gkw7 krb5kdc[17773](info): commencing operation
krb5kdc: starting...
W20260807 18:18:23.020550 19048 mini_kdc.cc:121] Time spent starting KDC: real 1.290s user 0.004s sys 0.044s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786126328808105-19048-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786126328808105-19048-0/krb5kdc/test-user.keytab.
Aug 07 18:18:23 dist-test-slave-gkw7 krb5kdc[17773](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126703, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:18:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:18:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.154.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.154.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.154.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.62.keytab.
Entry for principal kudu/127.18.154.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.62.keytab.
I20260807 18:18:23.356743 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786126328808105-19048-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786126328808105-19048-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786126328808105-19048-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.154.62:33165
--webserver_interface=127.18.154.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.62.keytab
--principal=kudu/127.18.154.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.154.20:42705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.154.62:33165
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786126328808105-19048-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786126328808105-19048-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786126328808105-19048-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:18:23.709764 17796 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:18:23.710140 17796 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:18:23.710317 17796 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:18:23.717640 17796 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:18:23.717818 17796 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:18:23.717934 17796 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:18:23.718046 17796 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:18:23.718135 17796 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:18:23.728355 17796 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.154.20:42705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786126328808105-19048-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786126328808105-19048-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.154.62:33165
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.62.keytab
--principal=kudu/127.18.154.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.154.62:33165
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.154.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786126328808105-19048-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:18:23.729841 17796 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:18:23.731251 17796 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:18:23.743789 17805 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:18:23.743876 17806 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:18:23.743901 17809 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:18:23.745013 17796 server_base.cc:1034] running on GCE node
I20260807 18:18:23.745918 17796 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:18:23.747383 17796 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:18:23.748591 17796 hybrid_clock.cc:648] HybridClock initialized: now 1786126703748549 us; error 32 us; skew 500 ppm
I20260807 18:18:23.748984 17796 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:18:23 dist-test-slave-gkw7 krb5kdc[17773](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126703, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:18:23.752394 17796 init.cc:377] Logged in from keytab as kudu/127.18.154.62@KRBTEST.COM (short username kudu)
I20260807 18:18:23.754086 17796 webserver.cc:460] Webserver started at http://127.18.154.62:38541/ using document root <none> and password file <none>
I20260807 18:18:23.754837 17796 fs_manager.cc:359] Metadata directory not provided
I20260807 18:18:23.754976 17796 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:18:23.755316 17796 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:18:23.758563 17796 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786126328808105-19048-0/minicluster-data/master-0/data/instance:
uuid: "85d3e40c970e4595b096aef8ac68fb98"
format_stamp: "Formatted at 2026-08-07 18:18:23 on dist-test-slave-gkw7"
server_key: "ff420e443d787ea8b52d3f09982e2261"
server_key_iv: "cca14e1a5219da8ad6d825412ccce88c"
server_key_version: "encryptionkey@0"
I20260807 18:18:23.759394 17796 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786126328808105-19048-0/minicluster-data/master-0/wal/instance:
uuid: "85d3e40c970e4595b096aef8ac68fb98"
format_stamp: "Formatted at 2026-08-07 18:18:23 on dist-test-slave-gkw7"
server_key: "ff420e443d787ea8b52d3f09982e2261"
server_key_iv: "cca14e1a5219da8ad6d825412ccce88c"
server_key_version: "encryptionkey@0"
I20260807 18:18:23.765019 17796 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260807 18:18:23.768774 17817 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:18:23.770300 17796 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:18:23.770489 17796 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786126328808105-19048-0/minicluster-data/master-0/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786126328808105-19048-0/minicluster-data/master-0/wal
uuid: "85d3e40c970e4595b096aef8ac68fb98"
format_stamp: "Formatted at 2026-08-07 18:18:23 on dist-test-slave-gkw7"
server_key: "ff420e443d787ea8b52d3f09982e2261"
server_key_iv: "cca14e1a5219da8ad6d825412ccce88c"
server_key_version: "encryptionkey@0"
I20260807 18:18:23.770689 17796 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786126328808105-19048-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786126328808105-19048-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786126328808105-19048-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:18:23.799121 17796 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:18:23.803792 17796 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:18:23.804119 17796 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:18:23.823979 17796 rpc_server.cc:307] RPC server started. Bound to: 127.18.154.62:33165
I20260807 18:18:23.824000 17888 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.154.62:33165 every 8 connection(s)
I20260807 18:18:23.825456 17796 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
I20260807 18:18:23.829952 17890 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:18:23.831563 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 17796
I20260807 18:18:23.831777 19048 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786126328808105-19048-0/minicluster-data/master-0/wal/instance
I20260807 18:18:23.832458 19048 external_mini_cluster.cc:1428] Setting key d568246e175254829f071523b204084b
I20260807 18:18:23.846983 17890 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 85d3e40c970e4595b096aef8ac68fb98: Bootstrap starting.
I20260807 18:18:23.849748 17894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:23.834859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53773 (local address 127.18.154.62:33165)
0807 18:18:23.836708 (+ 1849us) server_negotiation.cc:206] Beginning negotiation
0807 18:18:23.836727 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0807 18:18:23.836782 (+ 55us) server_negotiation.cc:407] Connection header received
0807 18:18:23.838857 (+ 2075us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:18:23.838891 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:18:23.839510 (+ 619us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:18:23.839968 (+ 458us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:18:23.840515 (+ 547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:23.842748 (+ 2233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:23.844101 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:23.845235 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:23.845429 (+ 194us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:18:23.846672 (+ 1243us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:18:23.846688 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:18:23.846707 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:18:23.847493 (+ 786us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:18:23.847608 (+ 115us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:18:23.847642 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:18:23.847807 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0807 18:18:23.848082 (+ 275us) server_negotiation.cc:299] Negotiation successful
0807 18:18:23.848260 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1547,"thread_start_us":494,"threads_started":1}
I20260807 18:18:23.851193 17890 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 85d3e40c970e4595b096aef8ac68fb98: Neither blocks nor log segments found. Creating new log.
I20260807 18:18:23.852420 17890 log.cc:824] T 00000000000000000000000000000000 P 85d3e40c970e4595b096aef8ac68fb98: Log is configured to *not* fsync() on all Append() calls
I20260807 18:18:23.853823 19048 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:18:23.855350 17890 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 85d3e40c970e4595b096aef8ac68fb98: No bootstrap required, opened a new log
I20260807 18:18:23.864225 17890 raft_consensus.cc:348] T 00000000000000000000000000000000 P 85d3e40c970e4595b096aef8ac68fb98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85d3e40c970e4595b096aef8ac68fb98" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 33165 } }
I20260807 18:18:23.864679 17890 raft_consensus.cc:374] T 00000000000000000000000000000000 P 85d3e40c970e4595b096aef8ac68fb98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:18:23.864786 17890 raft_consensus.cc:729] T 00000000000000000000000000000000 P 85d3e40c970e4595b096aef8ac68fb98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85d3e40c970e4595b096aef8ac68fb98, State: Initialized, Role: FOLLOWER
I20260807 18:18:23.865464 17890 consensus_queue.cc:260] T 00000000000000000000000000000000 P 85d3e40c970e4595b096aef8ac68fb98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85d3e40c970e4595b096aef8ac68fb98" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 33165 } }
I20260807 18:18:23.865816 17890 raft_consensus.cc:388] T 00000000000000000000000000000000 P 85d3e40c970e4595b096aef8ac68fb98 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:18:23.865978 17890 raft_consensus.cc:482] T 00000000000000000000000000000000 P 85d3e40c970e4595b096aef8ac68fb98 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:18:23.866232 17890 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 85d3e40c970e4595b096aef8ac68fb98 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:18:23.866369 17894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:23.858878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39051 (local address 127.18.154.62:33165)
0807 18:18:23.859067 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0807 18:18:23.859074 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:18:23.859559 (+ 485us) server_negotiation.cc:407] Connection header received
0807 18:18:23.859724 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:18:23.859730 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:18:23.859813 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:18:23.859937 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:18:23.860544 (+ 607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:23.861801 (+ 1257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:23.863288 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:23.863910 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:23.864048 (+ 138us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:18:23.865324 (+ 1276us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:18:23.865330 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:18:23.865334 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:18:23.865361 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:18:23.865458 (+ 97us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:18:23.865478 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:18:23.865717 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0807 18:18:23.865910 (+ 193us) server_negotiation.cc:299] Negotiation successful
0807 18:18:23.865978 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260807 18:18:23.868775 17890 raft_consensus.cc:504] T 00000000000000000000000000000000 P 85d3e40c970e4595b096aef8ac68fb98 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85d3e40c970e4595b096aef8ac68fb98" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 33165 } }
I20260807 18:18:23.869256 17890 leader_election.cc:302] T 00000000000000000000000000000000 P 85d3e40c970e4595b096aef8ac68fb98 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 85d3e40c970e4595b096aef8ac68fb98; no voters:
I20260807 18:18:23.869910 17890 leader_election.cc:288] T 00000000000000000000000000000000 P 85d3e40c970e4595b096aef8ac68fb98 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:18:23.870113 17904 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 85d3e40c970e4595b096aef8ac68fb98 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:18:23.871297 17904 raft_consensus.cc:686] T 00000000000000000000000000000000 P 85d3e40c970e4595b096aef8ac68fb98 [term 1 LEADER]: Becoming Leader. State: Replica: 85d3e40c970e4595b096aef8ac68fb98, State: Running, Role: LEADER
I20260807 18:18:23.872009 17904 consensus_queue.cc:237] T 00000000000000000000000000000000 P 85d3e40c970e4595b096aef8ac68fb98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85d3e40c970e4595b096aef8ac68fb98" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 33165 } }
I20260807 18:18:23.872360 17890 sys_catalog.cc:564] T 00000000000000000000000000000000 P 85d3e40c970e4595b096aef8ac68fb98 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:18:23.875447 17906 sys_catalog.cc:455] T 00000000000000000000000000000000 P 85d3e40c970e4595b096aef8ac68fb98 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 85d3e40c970e4595b096aef8ac68fb98. Latest consensus state: current_term: 1 leader_uuid: "85d3e40c970e4595b096aef8ac68fb98" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85d3e40c970e4595b096aef8ac68fb98" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 33165 } } }
I20260807 18:18:23.875900 17906 sys_catalog.cc:458] T 00000000000000000000000000000000 P 85d3e40c970e4595b096aef8ac68fb98 [sys.catalog]: This master's current role is: LEADER
I20260807 18:18:23.875412 17905 sys_catalog.cc:455] T 00000000000000000000000000000000 P 85d3e40c970e4595b096aef8ac68fb98 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "85d3e40c970e4595b096aef8ac68fb98" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85d3e40c970e4595b096aef8ac68fb98" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 33165 } } }
I20260807 18:18:23.876385 17905 sys_catalog.cc:458] T 00000000000000000000000000000000 P 85d3e40c970e4595b096aef8ac68fb98 [sys.catalog]: This master's current role is: LEADER
I20260807 18:18:23.882597 17911 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:18:23.887142 17911 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:18:23.896390 17911 catalog_manager.cc:1347] Generated new cluster ID: 4e3b2c9595fe45d5bba6e394e18cbeb0
I20260807 18:18:23.896521 17911 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:18:23.913482 17911 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:18:23.915370 17911 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:18:23.923138 17911 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 85d3e40c970e4595b096aef8ac68fb98: Generated new TSK 0
I20260807 18:18:23.924203 17911 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:18:23.932874 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 17796
W20260807 18:18:23.934489 17882 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:53773
W20260807 18:18:23.935341 19048 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 17796: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 18:18:23.935527 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 17796
W20260807 18:18:23.965406 17771 connection.cc:535] client connection to 127.18.154.62:33165 recv error: Network error: failed to read from TLS socket (remote: 127.18.154.62:33165): Connection reset by peer (error 104)
2026-08-07T18:18:23Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (2254 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786126328808105-19048-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:18:24 dist-test-slave-gkw7 krb5kdc[17930](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:18:24 dist-test-slave-gkw7 krb5kdc[17930](info): set up 2 sockets
Aug 07 18:18:24 dist-test-slave-gkw7 krb5kdc[17930](info): commencing operation
krb5kdc: starting...
W20260807 18:18:25.313321 19048 mini_kdc.cc:121] Time spent starting KDC: real 1.329s user 0.005s sys 0.041s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786126328808105-19048-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786126328808105-19048-0/krb5kdc/test-user.keytab.
Aug 07 18:18:25 dist-test-slave-gkw7 krb5kdc[17930](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126705, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:18:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:18:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.154.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.154.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.154.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.62.keytab.
Entry for principal kudu/127.18.154.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.62.keytab.
I20260807 18:18:25.649366 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786126328808105-19048-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786126328808105-19048-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786126328808105-19048-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.154.62:44683
--webserver_interface=127.18.154.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.62.keytab
--principal=kudu/127.18.154.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.154.20:33081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.154.62:44683
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786126328808105-19048-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786126328808105-19048-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786126328808105-19048-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:18:26.006983 17948 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:18:26.007390 17948 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:18:26.007565 17948 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:18:26.015370 17948 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:18:26.015543 17948 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:18:26.015648 17948 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:18:26.015767 17948 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:18:26.015846 17948 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:18:26.026394 17948 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.154.20:33081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786126328808105-19048-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786126328808105-19048-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.154.62:44683
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.62.keytab
--principal=kudu/127.18.154.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.154.62:44683
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.154.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786126328808105-19048-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:18:26.027854 17948 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:18:26.029297 17948 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:18:26.042179 17954 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:18:26.042277 17957 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:18:26.042171 17955 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:18:26.043110 17948 server_base.cc:1034] running on GCE node
I20260807 18:18:26.043956 17948 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:18:26.045454 17948 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:18:26.046754 17948 hybrid_clock.cc:648] HybridClock initialized: now 1786126706046688 us; error 59 us; skew 500 ppm
I20260807 18:18:26.047194 17948 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:18:26 dist-test-slave-gkw7 krb5kdc[17930](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126706, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:18:26.056799 17948 init.cc:377] Logged in from keytab as kudu/127.18.154.62@KRBTEST.COM (short username kudu)
I20260807 18:18:26.059015 17948 webserver.cc:460] Webserver started at http://127.18.154.62:33923/ using document root <none> and password file <none>
I20260807 18:18:26.059991 17948 fs_manager.cc:359] Metadata directory not provided
I20260807 18:18:26.060204 17948 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:18:26.060638 17948 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:18:26.064546 17948 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786126328808105-19048-0/minicluster-data/master-0/data/instance:
uuid: "33b2099d9f9d4f758dce8e2515fd7e2d"
format_stamp: "Formatted at 2026-08-07 18:18:26 on dist-test-slave-gkw7"
server_key: "386e379a1a96779b93868017363bc52e"
server_key_iv: "4af53fa4c49e6c012480ac5c9b34b459"
server_key_version: "encryptionkey@0"
I20260807 18:18:26.065623 17948 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786126328808105-19048-0/minicluster-data/master-0/wal/instance:
uuid: "33b2099d9f9d4f758dce8e2515fd7e2d"
format_stamp: "Formatted at 2026-08-07 18:18:26 on dist-test-slave-gkw7"
server_key: "386e379a1a96779b93868017363bc52e"
server_key_iv: "4af53fa4c49e6c012480ac5c9b34b459"
server_key_version: "encryptionkey@0"
I20260807 18:18:26.072355 17948 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.001s sys 0.004s
I20260807 18:18:26.076972 17967 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:18:26.078774 17948 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260807 18:18:26.079000 17948 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786126328808105-19048-0/minicluster-data/master-0/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786126328808105-19048-0/minicluster-data/master-0/wal
uuid: "33b2099d9f9d4f758dce8e2515fd7e2d"
format_stamp: "Formatted at 2026-08-07 18:18:26 on dist-test-slave-gkw7"
server_key: "386e379a1a96779b93868017363bc52e"
server_key_iv: "4af53fa4c49e6c012480ac5c9b34b459"
server_key_version: "encryptionkey@0"
I20260807 18:18:26.079250 17948 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786126328808105-19048-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786126328808105-19048-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786126328808105-19048-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:18:26.085865 17948 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:18:26.090629 17948 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:18:26.090972 17948 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:18:26.110321 17948 rpc_server.cc:307] RPC server started. Bound to: 127.18.154.62:44683
I20260807 18:18:26.110358 18042 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.154.62:44683 every 8 connection(s)
I20260807 18:18:26.111891 17948 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
I20260807 18:18:26.113693 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 17948
I20260807 18:18:26.113898 19048 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786126328808105-19048-0/minicluster-data/master-0/wal/instance
I20260807 18:18:26.114470 19048 external_mini_cluster.cc:1428] Setting key 12441db030bc5db1b9acaa3d1c11ef04
I20260807 18:18:26.116798 18043 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:18:26.122624 18046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:26.116388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50185 (local address 127.18.154.62:44683)
0807 18:18:26.117256 (+ 868us) server_negotiation.cc:206] Beginning negotiation
0807 18:18:26.117269 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 18:18:26.117314 (+ 45us) server_negotiation.cc:407] Connection header received
0807 18:18:26.119059 (+ 1745us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:18:26.119083 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:18:26.119453 (+ 370us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:18:26.119800 (+ 347us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:18:26.120899 (+ 1099us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:18:26.120908 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:18:26.120922 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:18:26.120936 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:18:26.121020 (+ 84us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:18:26.121041 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:18:26.121271 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0807 18:18:26.121370 (+ 99us) server_negotiation.cc:299] Negotiation successful
0807 18:18:26.121625 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":628,"thread_start_us":406,"threads_started":1}
I20260807 18:18:26.126639 19048 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:18:26.129638 18043 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 33b2099d9f9d4f758dce8e2515fd7e2d: Bootstrap starting.
I20260807 18:18:26.133199 18043 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 33b2099d9f9d4f758dce8e2515fd7e2d: Neither blocks nor log segments found. Creating new log.
I20260807 18:18:26.134109 18046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:26.131178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43429 (local address 127.18.154.62:44683)
0807 18:18:26.131395 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0807 18:18:26.131403 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:18:26.131801 (+ 398us) server_negotiation.cc:407] Connection header received
0807 18:18:26.132007 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:18:26.132012 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:18:26.132072 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:18:26.132226 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:18:26.133256 (+ 1030us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:18:26.133262 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:18:26.133266 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:18:26.133284 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:18:26.133358 (+ 74us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:18:26.133379 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:18:26.133492 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0807 18:18:26.133841 (+ 349us) server_negotiation.cc:299] Negotiation successful
0807 18:18:26.133891 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260807 18:18:26.134586 18043 log.cc:824] T 00000000000000000000000000000000 P 33b2099d9f9d4f758dce8e2515fd7e2d: Log is configured to *not* fsync() on all Append() calls
I20260807 18:18:26.137683 18043 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 33b2099d9f9d4f758dce8e2515fd7e2d: No bootstrap required, opened a new log
I20260807 18:18:26.145020 18043 raft_consensus.cc:348] T 00000000000000000000000000000000 P 33b2099d9f9d4f758dce8e2515fd7e2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33b2099d9f9d4f758dce8e2515fd7e2d" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 44683 } }
I20260807 18:18:26.145450 18043 raft_consensus.cc:374] T 00000000000000000000000000000000 P 33b2099d9f9d4f758dce8e2515fd7e2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:18:26.145584 18043 raft_consensus.cc:729] T 00000000000000000000000000000000 P 33b2099d9f9d4f758dce8e2515fd7e2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33b2099d9f9d4f758dce8e2515fd7e2d, State: Initialized, Role: FOLLOWER
I20260807 18:18:26.146297 18043 consensus_queue.cc:260] T 00000000000000000000000000000000 P 33b2099d9f9d4f758dce8e2515fd7e2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33b2099d9f9d4f758dce8e2515fd7e2d" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 44683 } }
I20260807 18:18:26.146554 18043 raft_consensus.cc:388] T 00000000000000000000000000000000 P 33b2099d9f9d4f758dce8e2515fd7e2d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:18:26.146708 18043 raft_consensus.cc:482] T 00000000000000000000000000000000 P 33b2099d9f9d4f758dce8e2515fd7e2d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:18:26.146894 18043 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 33b2099d9f9d4f758dce8e2515fd7e2d [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:18:26.149112 18043 raft_consensus.cc:504] T 00000000000000000000000000000000 P 33b2099d9f9d4f758dce8e2515fd7e2d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33b2099d9f9d4f758dce8e2515fd7e2d" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 44683 } }
I20260807 18:18:26.149760 18043 leader_election.cc:302] T 00000000000000000000000000000000 P 33b2099d9f9d4f758dce8e2515fd7e2d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 33b2099d9f9d4f758dce8e2515fd7e2d; no voters:
I20260807 18:18:26.150494 18043 leader_election.cc:288] T 00000000000000000000000000000000 P 33b2099d9f9d4f758dce8e2515fd7e2d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:18:26.150707 18054 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 33b2099d9f9d4f758dce8e2515fd7e2d [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:18:26.151536 18054 raft_consensus.cc:686] T 00000000000000000000000000000000 P 33b2099d9f9d4f758dce8e2515fd7e2d [term 1 LEADER]: Becoming Leader. State: Replica: 33b2099d9f9d4f758dce8e2515fd7e2d, State: Running, Role: LEADER
I20260807 18:18:26.152273 18054 consensus_queue.cc:237] T 00000000000000000000000000000000 P 33b2099d9f9d4f758dce8e2515fd7e2d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33b2099d9f9d4f758dce8e2515fd7e2d" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 44683 } }
I20260807 18:18:26.152812 18043 sys_catalog.cc:564] T 00000000000000000000000000000000 P 33b2099d9f9d4f758dce8e2515fd7e2d [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:18:26.155494 18055 sys_catalog.cc:455] T 00000000000000000000000000000000 P 33b2099d9f9d4f758dce8e2515fd7e2d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "33b2099d9f9d4f758dce8e2515fd7e2d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33b2099d9f9d4f758dce8e2515fd7e2d" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 44683 } } }
I20260807 18:18:26.155995 18055 sys_catalog.cc:458] T 00000000000000000000000000000000 P 33b2099d9f9d4f758dce8e2515fd7e2d [sys.catalog]: This master's current role is: LEADER
I20260807 18:18:26.156531 18056 sys_catalog.cc:455] T 00000000000000000000000000000000 P 33b2099d9f9d4f758dce8e2515fd7e2d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 33b2099d9f9d4f758dce8e2515fd7e2d. Latest consensus state: current_term: 1 leader_uuid: "33b2099d9f9d4f758dce8e2515fd7e2d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33b2099d9f9d4f758dce8e2515fd7e2d" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 44683 } } }
I20260807 18:18:26.156878 18056 sys_catalog.cc:458] T 00000000000000000000000000000000 P 33b2099d9f9d4f758dce8e2515fd7e2d [sys.catalog]: This master's current role is: LEADER
I20260807 18:18:26.158851 18062 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:18:26.164482 18062 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:18:26.174436 18062 catalog_manager.cc:1347] Generated new cluster ID: b0b222dc06d94d1bacb8c88a99733d9a
I20260807 18:18:26.174558 18062 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:18:26.194304 18062 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:18:26.194839 18062 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:18:26.213147 18062 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 33b2099d9f9d4f758dce8e2515fd7e2d: Generated new TSK 0
I20260807 18:18:26.214247 18062 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:18:26.223910 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 17948
W20260807 18:18:26.225390 18038 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:50185
W20260807 18:18:26.226243 19048 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 17948: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 18:18:26.226428 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 17948
2026-08-07T18:18:26Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (2289 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786126328808105-19048-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:18:26 dist-test-slave-gkw7 krb5kdc[18076](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:18:26 dist-test-slave-gkw7 krb5kdc[18076](info): set up 2 sockets
Aug 07 18:18:26 dist-test-slave-gkw7 krb5kdc[18076](info): commencing operation
krb5kdc: starting...
W20260807 18:18:27.582970 19048 mini_kdc.cc:121] Time spent starting KDC: real 1.308s user 0.001s sys 0.046s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786126328808105-19048-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786126328808105-19048-0/krb5kdc/test-user.keytab.
Aug 07 18:18:27 dist-test-slave-gkw7 krb5kdc[18076](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126707, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:18:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:18:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.154.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.154.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.154.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.62.keytab.
Entry for principal kudu/127.18.154.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.62.keytab.
I20260807 18:18:27.923115 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786126328808105-19048-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786126328808105-19048-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786126328808105-19048-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.154.62:40601
--webserver_interface=127.18.154.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.62.keytab
--principal=kudu/127.18.154.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.154.20:45917
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.154.62:40601
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786126328808105-19048-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786126328808105-19048-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786126328808105-19048-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:18:28.272071 18097 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:18:28.272434 18097 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:18:28.272673 18097 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:18:28.279958 18097 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:18:28.280061 18097 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:18:28.280169 18097 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:18:28.280277 18097 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:18:28.280342 18097 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:18:28.290266 18097 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.154.20:45917
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786126328808105-19048-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786126328808105-19048-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.154.62:40601
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.62.keytab
--principal=kudu/127.18.154.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.154.62:40601
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.154.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786126328808105-19048-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:18:28.291597 18097 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:18:28.292991 18097 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:18:28.304981 18108 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:18:28.305042 18111 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:18:28.305065 18109 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:18:28.305991 18097 server_base.cc:1034] running on GCE node
I20260807 18:18:28.306710 18097 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:18:28.308014 18097 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:18:28.309213 18097 hybrid_clock.cc:648] HybridClock initialized: now 1786126708309150 us; error 57 us; skew 500 ppm
I20260807 18:18:28.309631 18097 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:18:28 dist-test-slave-gkw7 krb5kdc[18076](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126708, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:18:28.312770 18097 init.cc:377] Logged in from keytab as kudu/127.18.154.62@KRBTEST.COM (short username kudu)
I20260807 18:18:28.314289 18097 webserver.cc:460] Webserver started at http://127.18.154.62:41811/ using document root <none> and password file <none>
I20260807 18:18:28.315007 18097 fs_manager.cc:359] Metadata directory not provided
I20260807 18:18:28.315162 18097 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:18:28.315474 18097 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:18:28.318436 18097 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786126328808105-19048-0/minicluster-data/master-0/data/instance:
uuid: "b9308284749d49908fc8f4a4c8465889"
format_stamp: "Formatted at 2026-08-07 18:18:28 on dist-test-slave-gkw7"
server_key: "bbf4eb0acb371b7170fe1f02e5e1885b"
server_key_iv: "2af57422fb3b32e8726835843169b06e"
server_key_version: "encryptionkey@0"
I20260807 18:18:28.319235 18097 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786126328808105-19048-0/minicluster-data/master-0/wal/instance:
uuid: "b9308284749d49908fc8f4a4c8465889"
format_stamp: "Formatted at 2026-08-07 18:18:28 on dist-test-slave-gkw7"
server_key: "bbf4eb0acb371b7170fe1f02e5e1885b"
server_key_iv: "2af57422fb3b32e8726835843169b06e"
server_key_version: "encryptionkey@0"
I20260807 18:18:28.324420 18097 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260807 18:18:28.327913 18121 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:18:28.329344 18097 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260807 18:18:28.329569 18097 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786126328808105-19048-0/minicluster-data/master-0/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786126328808105-19048-0/minicluster-data/master-0/wal
uuid: "b9308284749d49908fc8f4a4c8465889"
format_stamp: "Formatted at 2026-08-07 18:18:28 on dist-test-slave-gkw7"
server_key: "bbf4eb0acb371b7170fe1f02e5e1885b"
server_key_iv: "2af57422fb3b32e8726835843169b06e"
server_key_version: "encryptionkey@0"
I20260807 18:18:28.329819 18097 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786126328808105-19048-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786126328808105-19048-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786126328808105-19048-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:18:28.355120 18097 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:18:28.359247 18097 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:18:28.359545 18097 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:18:28.378728 18097 rpc_server.cc:307] RPC server started. Bound to: 127.18.154.62:40601
I20260807 18:18:28.378767 18198 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.154.62:40601 every 8 connection(s)
I20260807 18:18:28.380271 18097 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
I20260807 18:18:28.384785 18199 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:18:28.385921 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 18097
I20260807 18:18:28.386075 19048 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786126328808105-19048-0/minicluster-data/master-0/wal/instance
I20260807 18:18:28.386466 19048 external_mini_cluster.cc:1428] Setting key 91dec120e11d315b5ad43528cfcba271
I20260807 18:18:28.397709 18199 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b9308284749d49908fc8f4a4c8465889: Bootstrap starting.
Aug 07 18:18:28 dist-test-slave-gkw7 krb5kdc[18076](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126707, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.154.62@KRBTEST.COM
I20260807 18:18:28.400760 18199 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b9308284749d49908fc8f4a4c8465889: Neither blocks nor log segments found. Creating new log.
I20260807 18:18:28.401895 18199 log.cc:824] T 00000000000000000000000000000000 P b9308284749d49908fc8f4a4c8465889: Log is configured to *not* fsync() on all Append() calls
I20260807 18:18:28.405045 18199 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b9308284749d49908fc8f4a4c8465889: No bootstrap required, opened a new log
I20260807 18:18:28.409478 18203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:28.388327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37485 (local address 127.18.154.62:40601)
0807 18:18:28.389142 (+ 815us) server_negotiation.cc:206] Beginning negotiation
0807 18:18:28.389157 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 18:18:28.389199 (+ 42us) server_negotiation.cc:407] Connection header received
0807 18:18:28.391388 (+ 2189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:18:28.391412 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:18:28.391836 (+ 424us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:18:28.392244 (+ 408us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:18:28.393825 (+ 1581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:28.395224 (+ 1399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:28.396606 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:28.397273 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:28.397480 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:18:28.401674 (+ 4194us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:18:28.401683 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:18:28.401701 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:18:28.401737 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:18:28.405225 (+ 3488us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:18:28.405987 (+ 762us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:18:28.405996 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:18:28.406005 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:18:28.406196 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:18:28.406608 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:18:28.406613 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:18:28.406616 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:18:28.407182 (+ 566us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:18:28.407506 (+ 324us) server_negotiation.cc:1035] Waiting for connection context
0807 18:18:28.408010 (+ 504us) server_negotiation.cc:299] Negotiation successful
0807 18:18:28.408468 (+ 458us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":584,"thread_start_us":328,"threads_started":1}
I20260807 18:18:28.413506 19048 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:18:28.413914 18199 raft_consensus.cc:348] T 00000000000000000000000000000000 P b9308284749d49908fc8f4a4c8465889 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9308284749d49908fc8f4a4c8465889" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 40601 } }
I20260807 18:18:28.414273 18199 raft_consensus.cc:374] T 00000000000000000000000000000000 P b9308284749d49908fc8f4a4c8465889 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:18:28.414386 18199 raft_consensus.cc:729] T 00000000000000000000000000000000 P b9308284749d49908fc8f4a4c8465889 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b9308284749d49908fc8f4a4c8465889, State: Initialized, Role: FOLLOWER
I20260807 18:18:28.414994 18199 consensus_queue.cc:260] T 00000000000000000000000000000000 P b9308284749d49908fc8f4a4c8465889 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9308284749d49908fc8f4a4c8465889" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 40601 } }
I20260807 18:18:28.415235 18199 raft_consensus.cc:388] T 00000000000000000000000000000000 P b9308284749d49908fc8f4a4c8465889 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:18:28.415395 18199 raft_consensus.cc:482] T 00000000000000000000000000000000 P b9308284749d49908fc8f4a4c8465889 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:18:28.415580 18199 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b9308284749d49908fc8f4a4c8465889 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:18:28.418277 18199 raft_consensus.cc:504] T 00000000000000000000000000000000 P b9308284749d49908fc8f4a4c8465889 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9308284749d49908fc8f4a4c8465889" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 40601 } }
I20260807 18:18:28.418828 18199 leader_election.cc:302] T 00000000000000000000000000000000 P b9308284749d49908fc8f4a4c8465889 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b9308284749d49908fc8f4a4c8465889; no voters:
I20260807 18:18:28.419554 18199 leader_election.cc:288] T 00000000000000000000000000000000 P b9308284749d49908fc8f4a4c8465889 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:18:28.419734 18213 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b9308284749d49908fc8f4a4c8465889 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:18:28.420441 18213 raft_consensus.cc:686] T 00000000000000000000000000000000 P b9308284749d49908fc8f4a4c8465889 [term 1 LEADER]: Becoming Leader. State: Replica: b9308284749d49908fc8f4a4c8465889, State: Running, Role: LEADER
I20260807 18:18:28.421047 18213 consensus_queue.cc:237] T 00000000000000000000000000000000 P b9308284749d49908fc8f4a4c8465889 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9308284749d49908fc8f4a4c8465889" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 40601 } }
I20260807 18:18:28.422987 18213 sys_catalog.cc:455] T 00000000000000000000000000000000 P b9308284749d49908fc8f4a4c8465889 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b9308284749d49908fc8f4a4c8465889. Latest consensus state: current_term: 1 leader_uuid: "b9308284749d49908fc8f4a4c8465889" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9308284749d49908fc8f4a4c8465889" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 40601 } } }
I20260807 18:18:28.423277 18213 sys_catalog.cc:458] T 00000000000000000000000000000000 P b9308284749d49908fc8f4a4c8465889 [sys.catalog]: This master's current role is: LEADER
I20260807 18:18:28.424139 18217 sys_catalog.cc:455] T 00000000000000000000000000000000 P b9308284749d49908fc8f4a4c8465889 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b9308284749d49908fc8f4a4c8465889" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9308284749d49908fc8f4a4c8465889" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 40601 } } }
I20260807 18:18:28.424561 18217 sys_catalog.cc:458] T 00000000000000000000000000000000 P b9308284749d49908fc8f4a4c8465889 [sys.catalog]: This master's current role is: LEADER
I20260807 18:18:28.428184 18199 sys_catalog.cc:564] T 00000000000000000000000000000000 P b9308284749d49908fc8f4a4c8465889 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:18:28.429358 18220 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:18:28.436076 18220 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:18:28.438275 18203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:28.418238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33837 (local address 127.18.154.62:40601)
0807 18:18:28.418436 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0807 18:18:28.418444 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:18:28.418908 (+ 464us) server_negotiation.cc:407] Connection header received
0807 18:18:28.419024 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:18:28.419030 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:18:28.419110 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:18:28.419230 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:18:28.421298 (+ 2068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:28.422839 (+ 1541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:28.425163 (+ 2324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:28.426184 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:28.426337 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:18:28.429914 (+ 3577us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:18:28.429928 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:18:28.429937 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:18:28.429992 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:18:28.434685 (+ 4693us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:18:28.435593 (+ 908us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:18:28.435598 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:18:28.435601 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:18:28.435766 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:18:28.436313 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:18:28.436318 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:18:28.436321 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:18:28.436728 (+ 407us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:18:28.437678 (+ 950us) server_negotiation.cc:1035] Waiting for connection context
0807 18:18:28.437845 (+ 167us) server_negotiation.cc:299] Negotiation successful
0807 18:18:28.438021 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260807 18:18:28.449062 18220 catalog_manager.cc:1347] Generated new cluster ID: 4941bb79b8834520b73010b94cc8d954
I20260807 18:18:28.449190 18220 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:18:28.460336 18220 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:18:28.462791 18220 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:18:28.470803 18220 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b9308284749d49908fc8f4a4c8465889: Generated new TSK 0
I20260807 18:18:28.471730 18220 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:18:28.480587 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 18097
I20260807 18:18:28.481742 18193 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:18:28.571568 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 18097
2026-08-07T18:18:28Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (2341 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786126328808105-19048-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:18:28 dist-test-slave-gkw7 krb5kdc[18247](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:18:28 dist-test-slave-gkw7 krb5kdc[18247](info): set up 2 sockets
Aug 07 18:18:28 dist-test-slave-gkw7 krb5kdc[18247](info): commencing operation
krb5kdc: starting...
W20260807 18:18:29.911856 19048 mini_kdc.cc:121] Time spent starting KDC: real 1.296s user 0.001s sys 0.046s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786126328808105-19048-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786126328808105-19048-0/krb5kdc/test-user.keytab.
Aug 07 18:18:30 dist-test-slave-gkw7 krb5kdc[18247](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126710, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:18:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:18:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.154.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.154.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.154.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.62.keytab.
Entry for principal kudu/127.18.154.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.62.keytab.
I20260807 18:18:30.236641 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786126328808105-19048-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786126328808105-19048-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786126328808105-19048-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.154.62:37511
--webserver_interface=127.18.154.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.62.keytab
--principal=kudu/127.18.154.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.154.20:40575
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.154.62:37511
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786126328808105-19048-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786126328808105-19048-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786126328808105-19048-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:18:30.591094 18262 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:18:30.591444 18262 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:18:30.591539 18262 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:18:30.598737 18262 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:18:30.598847 18262 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:18:30.598893 18262 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:18:30.598956 18262 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:18:30.599001 18262 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:18:30.608749 18262 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.154.20:40575
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786126328808105-19048-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786126328808105-19048-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.154.62:37511
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.62.keytab
--principal=kudu/127.18.154.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.154.62:37511
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.154.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786126328808105-19048-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:18:30.610052 18262 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:18:30.611433 18262 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:18:30.624024 18270 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:18:30.623994 18271 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:18:30.624085 18274 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:18:30.625211 18262 server_base.cc:1034] running on GCE node
I20260807 18:18:30.626076 18262 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:18:30.627578 18262 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:18:30.628834 18262 hybrid_clock.cc:648] HybridClock initialized: now 1786126710628786 us; error 41 us; skew 500 ppm
I20260807 18:18:30.629228 18262 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:18:30 dist-test-slave-gkw7 krb5kdc[18247](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126710, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:18:30.632735 18262 init.cc:377] Logged in from keytab as kudu/127.18.154.62@KRBTEST.COM (short username kudu)
I20260807 18:18:30.634382 18262 webserver.cc:460] Webserver started at http://127.18.154.62:35321/ using document root <none> and password file <none>
I20260807 18:18:30.635118 18262 fs_manager.cc:359] Metadata directory not provided
I20260807 18:18:30.635268 18262 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:18:30.635589 18262 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:18:30.638830 18262 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786126328808105-19048-0/minicluster-data/master-0/data/instance:
uuid: "6f57973f910549cd839ae7e230476806"
format_stamp: "Formatted at 2026-08-07 18:18:30 on dist-test-slave-gkw7"
server_key: "c8e9c99ec9f7cae19b2a32e3a86ace6f"
server_key_iv: "4f28aaacf59615b4a595fe9f9c40c8cb"
server_key_version: "encryptionkey@0"
I20260807 18:18:30.639629 18262 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786126328808105-19048-0/minicluster-data/master-0/wal/instance:
uuid: "6f57973f910549cd839ae7e230476806"
format_stamp: "Formatted at 2026-08-07 18:18:30 on dist-test-slave-gkw7"
server_key: "c8e9c99ec9f7cae19b2a32e3a86ace6f"
server_key_iv: "4f28aaacf59615b4a595fe9f9c40c8cb"
server_key_version: "encryptionkey@0"
I20260807 18:18:30.645080 18262 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 18:18:30.648865 18284 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:18:30.650367 18262 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:18:30.650597 18262 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786126328808105-19048-0/minicluster-data/master-0/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786126328808105-19048-0/minicluster-data/master-0/wal
uuid: "6f57973f910549cd839ae7e230476806"
format_stamp: "Formatted at 2026-08-07 18:18:30 on dist-test-slave-gkw7"
server_key: "c8e9c99ec9f7cae19b2a32e3a86ace6f"
server_key_iv: "4f28aaacf59615b4a595fe9f9c40c8cb"
server_key_version: "encryptionkey@0"
I20260807 18:18:30.650815 18262 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786126328808105-19048-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786126328808105-19048-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786126328808105-19048-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:18:30.664420 18262 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:18:30.669019 18262 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:18:30.669324 18262 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:18:30.689612 18262 rpc_server.cc:307] RPC server started. Bound to: 127.18.154.62:37511
I20260807 18:18:30.689632 18360 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.154.62:37511 every 8 connection(s)
I20260807 18:18:30.691390 18262 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
I20260807 18:18:30.696024 18362 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:18:30.699473 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 18262
I20260807 18:18:30.699687 19048 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786126328808105-19048-0/minicluster-data/master-0/wal/instance
I20260807 18:18:30.700273 19048 external_mini_cluster.cc:1428] Setting key e2c3e3b4e3dde0cbb10018c98240e445
I20260807 18:18:30.708926 18362 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f57973f910549cd839ae7e230476806: Bootstrap starting.
I20260807 18:18:30.712318 18362 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6f57973f910549cd839ae7e230476806: Neither blocks nor log segments found. Creating new log.
I20260807 18:18:30.713639 18362 log.cc:824] T 00000000000000000000000000000000 P 6f57973f910549cd839ae7e230476806: Log is configured to *not* fsync() on all Append() calls
I20260807 18:18:30.714768 18367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:30.702461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52585 (local address 127.18.154.62:37511)
0807 18:18:30.703270 (+ 809us) server_negotiation.cc:206] Beginning negotiation
0807 18:18:30.703282 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 18:18:30.703325 (+ 43us) server_negotiation.cc:407] Connection header received
0807 18:18:30.705141 (+ 1816us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:18:30.705186 (+ 45us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:18:30.705719 (+ 533us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:18:30.706138 (+ 419us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:18:30.706774 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:30.708160 (+ 1386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:30.709676 (+ 1516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:30.710374 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:30.710607 (+ 233us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:18:30.711905 (+ 1298us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:18:30.711913 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:18:30.711930 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:18:30.712552 (+ 622us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:18:30.712642 (+ 90us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:18:30.712677 (+ 35us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:18:30.712829 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0807 18:18:30.713059 (+ 230us) server_negotiation.cc:299] Negotiation successful
0807 18:18:30.713351 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":579,"thread_start_us":357,"threads_started":1}
I20260807 18:18:30.717060 18362 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f57973f910549cd839ae7e230476806: No bootstrap required, opened a new log
I20260807 18:18:30.718958 19048 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:18:30.725828 18362 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6f57973f910549cd839ae7e230476806 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f57973f910549cd839ae7e230476806" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 37511 } }
I20260807 18:18:30.726240 18362 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6f57973f910549cd839ae7e230476806 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:18:30.726377 18362 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6f57973f910549cd839ae7e230476806 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f57973f910549cd839ae7e230476806, State: Initialized, Role: FOLLOWER
I20260807 18:18:30.727062 18362 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6f57973f910549cd839ae7e230476806 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f57973f910549cd839ae7e230476806" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 37511 } }
I20260807 18:18:30.727332 18362 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6f57973f910549cd839ae7e230476806 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:18:30.727486 18362 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6f57973f910549cd839ae7e230476806 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:18:30.727674 18362 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6f57973f910549cd839ae7e230476806 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:18:30.730376 18367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:30.723639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43095 (local address 127.18.154.62:37511)
0807 18:18:30.723940 (+ 301us) server_negotiation.cc:206] Beginning negotiation
0807 18:18:30.723957 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 18:18:30.724325 (+ 368us) server_negotiation.cc:407] Connection header received
0807 18:18:30.724511 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:18:30.724515 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:18:30.724581 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:18:30.724670 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:18:30.725322 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:30.726119 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:30.727718 (+ 1599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:30.728220 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:30.728407 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:18:30.729635 (+ 1228us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:18:30.729640 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:18:30.729644 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:18:30.729662 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:18:30.729704 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:18:30.729719 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:18:30.729834 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0807 18:18:30.729969 (+ 135us) server_negotiation.cc:299] Negotiation successful
0807 18:18:30.730065 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260807 18:18:30.730554 18362 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6f57973f910549cd839ae7e230476806 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f57973f910549cd839ae7e230476806" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 37511 } }
I20260807 18:18:30.731284 18362 leader_election.cc:302] T 00000000000000000000000000000000 P 6f57973f910549cd839ae7e230476806 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6f57973f910549cd839ae7e230476806; no voters:
I20260807 18:18:30.732023 18362 leader_election.cc:288] T 00000000000000000000000000000000 P 6f57973f910549cd839ae7e230476806 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:18:30.732144 18380 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6f57973f910549cd839ae7e230476806 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:18:30.732903 18380 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6f57973f910549cd839ae7e230476806 [term 1 LEADER]: Becoming Leader. State: Replica: 6f57973f910549cd839ae7e230476806, State: Running, Role: LEADER
I20260807 18:18:30.733489 18380 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6f57973f910549cd839ae7e230476806 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f57973f910549cd839ae7e230476806" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 37511 } }
I20260807 18:18:30.734320 18362 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6f57973f910549cd839ae7e230476806 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:18:30.736573 18382 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f57973f910549cd839ae7e230476806 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6f57973f910549cd839ae7e230476806. Latest consensus state: current_term: 1 leader_uuid: "6f57973f910549cd839ae7e230476806" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f57973f910549cd839ae7e230476806" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 37511 } } }
I20260807 18:18:30.737041 18382 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f57973f910549cd839ae7e230476806 [sys.catalog]: This master's current role is: LEADER
I20260807 18:18:30.737946 18385 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:18:30.737757 18381 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f57973f910549cd839ae7e230476806 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6f57973f910549cd839ae7e230476806" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f57973f910549cd839ae7e230476806" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 37511 } } }
I20260807 18:18:30.740248 18381 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f57973f910549cd839ae7e230476806 [sys.catalog]: This master's current role is: LEADER
I20260807 18:18:30.743922 18385 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:18:30.753093 18385 catalog_manager.cc:1347] Generated new cluster ID: f0714ade64ce47339f50c7d5a7797645
I20260807 18:18:30.753203 18385 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:18:30.774300 18385 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:18:30.776232 18385 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:18:30.799190 18385 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6f57973f910549cd839ae7e230476806: Generated new TSK 0
I20260807 18:18:30.800376 18385 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:18:30.810544 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 18262
W20260807 18:18:30.811784 18355 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:52585
W20260807 18:18:30.812538 19048 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 18262: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 18:18:30.812706 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 18262
2026-08-07T18:18:30Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (2241 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786126328808105-19048-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:18:30 dist-test-slave-gkw7 krb5kdc[18407](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:18:30 dist-test-slave-gkw7 krb5kdc[18407](info): set up 2 sockets
Aug 07 18:18:30 dist-test-slave-gkw7 krb5kdc[18407](info): commencing operation
krb5kdc: starting...
W20260807 18:18:32.123908 19048 mini_kdc.cc:121] Time spent starting KDC: real 1.267s user 0.000s sys 0.045s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786126328808105-19048-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786126328808105-19048-0/krb5kdc/test-user.keytab.
Aug 07 18:18:32 dist-test-slave-gkw7 krb5kdc[18407](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126712, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:18:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:18:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.154.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.154.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.154.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.62.keytab.
Entry for principal kudu/127.18.154.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.62.keytab.
I20260807 18:18:32.453337 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786126328808105-19048-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786126328808105-19048-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786126328808105-19048-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.154.62:38109
--webserver_interface=127.18.154.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.62.keytab
--principal=kudu/127.18.154.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.154.20:45171
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.154.62:38109
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786126328808105-19048-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786126328808105-19048-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786126328808105-19048-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:18:32.813298 18424 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:18:32.813740 18424 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:18:32.813921 18424 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:18:32.821250 18424 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:18:32.821355 18424 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:18:32.821403 18424 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:18:32.821456 18424 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:18:32.821568 18424 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:18:32.831585 18424 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.154.20:45171
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786126328808105-19048-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786126328808105-19048-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.154.62:38109
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.62.keytab
--principal=kudu/127.18.154.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.154.62:38109
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.154.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786126328808105-19048-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:18:32.832968 18424 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:18:32.834784 18424 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:18:32.848188 18432 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:18:32.848292 18435 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:18:32.848243 18433 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:18:32.849182 18424 server_base.cc:1034] running on GCE node
I20260807 18:18:32.850085 18424 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:18:32.851545 18424 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:18:32.852766 18424 hybrid_clock.cc:648] HybridClock initialized: now 1786126712852715 us; error 39 us; skew 500 ppm
I20260807 18:18:32.853164 18424 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:18:32 dist-test-slave-gkw7 krb5kdc[18407](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126712, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:18:32.862768 18424 init.cc:377] Logged in from keytab as kudu/127.18.154.62@KRBTEST.COM (short username kudu)
I20260807 18:18:32.864564 18424 webserver.cc:460] Webserver started at http://127.18.154.62:44125/ using document root <none> and password file <none>
I20260807 18:18:32.865370 18424 fs_manager.cc:359] Metadata directory not provided
I20260807 18:18:32.865571 18424 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:18:32.865934 18424 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:18:32.869145 18424 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786126328808105-19048-0/minicluster-data/master-0/data/instance:
uuid: "2f3a989bc960459c82b8ef4cf5c048a5"
format_stamp: "Formatted at 2026-08-07 18:18:32 on dist-test-slave-gkw7"
server_key: "163235d9a9a24c8bcfe61a36e96f2167"
server_key_iv: "596c8e4b32cf7dc9a8c329f70884dede"
server_key_version: "encryptionkey@0"
I20260807 18:18:32.869978 18424 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786126328808105-19048-0/minicluster-data/master-0/wal/instance:
uuid: "2f3a989bc960459c82b8ef4cf5c048a5"
format_stamp: "Formatted at 2026-08-07 18:18:32 on dist-test-slave-gkw7"
server_key: "163235d9a9a24c8bcfe61a36e96f2167"
server_key_iv: "596c8e4b32cf7dc9a8c329f70884dede"
server_key_version: "encryptionkey@0"
I20260807 18:18:32.875429 18424 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.001s
I20260807 18:18:32.879269 18444 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:18:32.880738 18424 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 18:18:32.880961 18424 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786126328808105-19048-0/minicluster-data/master-0/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786126328808105-19048-0/minicluster-data/master-0/wal
uuid: "2f3a989bc960459c82b8ef4cf5c048a5"
format_stamp: "Formatted at 2026-08-07 18:18:32 on dist-test-slave-gkw7"
server_key: "163235d9a9a24c8bcfe61a36e96f2167"
server_key_iv: "596c8e4b32cf7dc9a8c329f70884dede"
server_key_version: "encryptionkey@0"
I20260807 18:18:32.881201 18424 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786126328808105-19048-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786126328808105-19048-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786126328808105-19048-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:18:32.888092 18424 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:18:32.893193 18424 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:18:32.893589 18424 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:18:32.915320 18424 rpc_server.cc:307] RPC server started. Bound to: 127.18.154.62:38109
I20260807 18:18:32.915343 18521 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.154.62:38109 every 8 connection(s)
I20260807 18:18:32.916992 18424 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
I20260807 18:18:32.920038 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 18424
I20260807 18:18:32.920274 19048 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786126328808105-19048-0/minicluster-data/master-0/wal/instance
I20260807 18:18:32.920885 19048 external_mini_cluster.cc:1428] Setting key 3c181ff3838866a1e5cc301cc3450b4d
I20260807 18:18:32.922097 18522 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:18:32.930665 18525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:32.922925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48201 (local address 127.18.154.62:38109)
0807 18:18:32.923856 (+ 931us) server_negotiation.cc:206] Beginning negotiation
0807 18:18:32.923872 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 18:18:32.923928 (+ 56us) server_negotiation.cc:407] Connection header received
0807 18:18:32.926187 (+ 2259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:18:32.926220 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:18:32.926737 (+ 517us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:18:32.927266 (+ 529us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:18:32.928383 (+ 1117us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:18:32.928394 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:18:32.928415 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:18:32.928442 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:18:32.928539 (+ 97us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:18:32.928569 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:18:32.928811 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0807 18:18:32.928963 (+ 152us) server_negotiation.cc:299] Negotiation successful
0807 18:18:32.929227 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":644,"thread_start_us":393,"threads_started":1}
I20260807 18:18:32.934897 19048 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:18:32.939321 18522 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2f3a989bc960459c82b8ef4cf5c048a5: Bootstrap starting.
I20260807 18:18:32.943806 18525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:32.940870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58563 (local address 127.18.154.62:38109)
0807 18:18:32.941090 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0807 18:18:32.941098 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:18:32.941563 (+ 465us) server_negotiation.cc:407] Connection header received
0807 18:18:32.941798 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:18:32.941804 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:18:32.941874 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:18:32.942018 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:18:32.943001 (+ 983us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:18:32.943007 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:18:32.943012 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:18:32.943032 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:18:32.943091 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:18:32.943113 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:18:32.943239 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0807 18:18:32.943363 (+ 124us) server_negotiation.cc:299] Negotiation successful
0807 18:18:32.943426 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260807 18:18:32.943816 18522 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2f3a989bc960459c82b8ef4cf5c048a5: Neither blocks nor log segments found. Creating new log.
I20260807 18:18:32.945472 18522 log.cc:824] T 00000000000000000000000000000000 P 2f3a989bc960459c82b8ef4cf5c048a5: Log is configured to *not* fsync() on all Append() calls
I20260807 18:18:32.948783 18522 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2f3a989bc960459c82b8ef4cf5c048a5: No bootstrap required, opened a new log
I20260807 18:18:32.957486 18522 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2f3a989bc960459c82b8ef4cf5c048a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f3a989bc960459c82b8ef4cf5c048a5" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 38109 } }
I20260807 18:18:32.958086 18522 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2f3a989bc960459c82b8ef4cf5c048a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:18:32.958207 18522 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2f3a989bc960459c82b8ef4cf5c048a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f3a989bc960459c82b8ef4cf5c048a5, State: Initialized, Role: FOLLOWER
I20260807 18:18:32.958956 18522 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2f3a989bc960459c82b8ef4cf5c048a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f3a989bc960459c82b8ef4cf5c048a5" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 38109 } }
I20260807 18:18:32.959206 18522 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2f3a989bc960459c82b8ef4cf5c048a5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:18:32.959375 18522 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2f3a989bc960459c82b8ef4cf5c048a5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:18:32.959558 18522 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2f3a989bc960459c82b8ef4cf5c048a5 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:18:32.961969 18522 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2f3a989bc960459c82b8ef4cf5c048a5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f3a989bc960459c82b8ef4cf5c048a5" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 38109 } }
I20260807 18:18:32.962535 18522 leader_election.cc:302] T 00000000000000000000000000000000 P 2f3a989bc960459c82b8ef4cf5c048a5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2f3a989bc960459c82b8ef4cf5c048a5; no voters:
I20260807 18:18:32.963201 18522 leader_election.cc:288] T 00000000000000000000000000000000 P 2f3a989bc960459c82b8ef4cf5c048a5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:18:32.963328 18537 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2f3a989bc960459c82b8ef4cf5c048a5 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:18:32.964701 18537 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2f3a989bc960459c82b8ef4cf5c048a5 [term 1 LEADER]: Becoming Leader. State: Replica: 2f3a989bc960459c82b8ef4cf5c048a5, State: Running, Role: LEADER
I20260807 18:18:32.965230 18537 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2f3a989bc960459c82b8ef4cf5c048a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f3a989bc960459c82b8ef4cf5c048a5" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 38109 } }
I20260807 18:18:32.965605 18522 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2f3a989bc960459c82b8ef4cf5c048a5 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:18:32.968588 18538 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2f3a989bc960459c82b8ef4cf5c048a5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2f3a989bc960459c82b8ef4cf5c048a5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f3a989bc960459c82b8ef4cf5c048a5" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 38109 } } }
I20260807 18:18:32.968672 18539 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2f3a989bc960459c82b8ef4cf5c048a5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2f3a989bc960459c82b8ef4cf5c048a5. Latest consensus state: current_term: 1 leader_uuid: "2f3a989bc960459c82b8ef4cf5c048a5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f3a989bc960459c82b8ef4cf5c048a5" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 38109 } } }
I20260807 18:18:32.969028 18538 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2f3a989bc960459c82b8ef4cf5c048a5 [sys.catalog]: This master's current role is: LEADER
I20260807 18:18:32.969033 18539 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2f3a989bc960459c82b8ef4cf5c048a5 [sys.catalog]: This master's current role is: LEADER
I20260807 18:18:32.972329 18550 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:18:32.976714 18550 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:18:32.986171 18550 catalog_manager.cc:1347] Generated new cluster ID: 23f113d2c1a440728064bccdfb96b909
I20260807 18:18:32.986326 18550 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:18:33.011394 18550 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:18:33.011998 18550 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:18:33.019958 18550 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2f3a989bc960459c82b8ef4cf5c048a5: Generated new TSK 0
I20260807 18:18:33.021091 18550 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:18:33.036170 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 18424
W20260807 18:18:33.037375 18514 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:48201
W20260807 18:18:33.038151 19048 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 18424: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 18:18:33.038318 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 18424
2026-08-07T18:18:33Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (2225 ms)
[----------] 18 tests from AuthTokenIssuingTest (42121 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126328808105-19048-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:18:33 dist-test-slave-gkw7 krb5kdc[18565](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:18:33 dist-test-slave-gkw7 krb5kdc[18565](info): set up 2 sockets
Aug 07 18:18:33 dist-test-slave-gkw7 krb5kdc[18565](info): commencing operation
krb5kdc: starting...
W20260807 18:18:34.366657 19048 mini_kdc.cc:121] Time spent starting KDC: real 1.283s user 0.002s sys 0.047s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126328808105-19048-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126328808105-19048-0/krb5kdc/test-user.keytab.
Aug 07 18:18:34 dist-test-slave-gkw7 krb5kdc[18565](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126714, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:18:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:18:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.154.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.154.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.154.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.62.keytab.
Entry for principal kudu/127.18.154.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.62.keytab.
I20260807 18:18:34.717435 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126328808105-19048-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126328808105-19048-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126328808105-19048-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.154.62:40517
--webserver_interface=127.18.154.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.62.keytab
--principal=kudu/127.18.154.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.154.20:37953
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.154.62:40517,127.18.154.61:44067,127.18.154.60:43209
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126328808105-19048-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126328808105-19048-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126328808105-19048-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:18:35.080034 18589 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 18:18:35.080423 18589 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:18:35.080557 18589 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:18:35.080663 18589 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:18:35.088124 18589 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:18:35.088224 18589 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:18:35.088318 18589 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:18:35.088398 18589 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:18:35.088521 18589 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:18:35.099211 18589 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.154.20:37953
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126328808105-19048-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126328808105-19048-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.154.62:40517,127.18.154.61:44067,127.18.154.60:43209
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.62.keytab
--principal=kudu/127.18.154.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.154.62:40517
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.154.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126328808105-19048-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:18:35.100672 18589 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:18:35.102136 18589 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:18:35.114869 18597 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:18:35.114977 18595 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:18:35.115553 18599 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:18:35.116091 18589 server_base.cc:1034] running on GCE node
I20260807 18:18:35.117014 18589 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:18:35.118690 18589 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:18:35.119956 18589 hybrid_clock.cc:648] HybridClock initialized: now 1786126715119870 us; error 77 us; skew 500 ppm
I20260807 18:18:35.120326 18589 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:18:35 dist-test-slave-gkw7 krb5kdc[18565](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126715, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:18:35.123754 18589 init.cc:377] Logged in from keytab as kudu/127.18.154.62@KRBTEST.COM (short username kudu)
I20260807 18:18:35.125283 18589 webserver.cc:460] Webserver started at http://127.18.154.62:36991/ using document root <none> and password file <none>
I20260807 18:18:35.126112 18589 fs_manager.cc:359] Metadata directory not provided
I20260807 18:18:35.126209 18589 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:18:35.126498 18589 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:18:35.129763 18589 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126328808105-19048-0/minicluster-data/master-0/data/instance:
uuid: "80ebcd8bbf9746a18b1dd35d401b7769"
format_stamp: "Formatted at 2026-08-07 18:18:35 on dist-test-slave-gkw7"
server_key: "19f9da1b0ddfe5ddf70943d3246a31d1"
server_key_iv: "558aa577e0fa586bf1360993b9752452"
server_key_version: "encryptionkey@0"
I20260807 18:18:35.130607 18589 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126328808105-19048-0/minicluster-data/master-0/wal/instance:
uuid: "80ebcd8bbf9746a18b1dd35d401b7769"
format_stamp: "Formatted at 2026-08-07 18:18:35 on dist-test-slave-gkw7"
server_key: "19f9da1b0ddfe5ddf70943d3246a31d1"
server_key_iv: "558aa577e0fa586bf1360993b9752452"
server_key_version: "encryptionkey@0"
I20260807 18:18:35.136358 18589 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 18:18:35.140074 18607 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:18:35.141750 18589 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260807 18:18:35.141971 18589 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126328808105-19048-0/minicluster-data/master-0/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126328808105-19048-0/minicluster-data/master-0/wal
uuid: "80ebcd8bbf9746a18b1dd35d401b7769"
format_stamp: "Formatted at 2026-08-07 18:18:35 on dist-test-slave-gkw7"
server_key: "19f9da1b0ddfe5ddf70943d3246a31d1"
server_key_iv: "558aa577e0fa586bf1360993b9752452"
server_key_version: "encryptionkey@0"
I20260807 18:18:35.142212 18589 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126328808105-19048-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126328808105-19048-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126328808105-19048-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:18:35.159549 18589 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:18:35.164266 18589 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:18:35.164567 18589 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:18:35.183817 18589 rpc_server.cc:307] RPC server started. Bound to: 127.18.154.62:40517
I20260807 18:18:35.183792 18677 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.154.62:40517 every 8 connection(s)
I20260807 18:18:35.185272 18589 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
I20260807 18:18:35.189262 18678 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:18:35.194135 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 18589
I20260807 18:18:35.194412 19048 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126328808105-19048-0/minicluster-data/master-0/wal/instance
I20260807 18:18:35.195209 19048 external_mini_cluster.cc:1428] Setting key 33d3f03127f5cff7dd2369f90e401bfb
I20260807 18:18:35.196262 18678 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.154.62" port: 40517 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 18:18:35 dist-test-slave-gkw7 krb5kdc[18565](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126715, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.62@KRBTEST.COM for kudu/127.18.154.62@KRBTEST.COM
I20260807 18:18:35.221340 18682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:35.199660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:40517 (local address 127.0.0.1:41622)
0807 18:18:35.200722 (+ 1062us) negotiation.cc:107] Waiting for socket to connect
0807 18:18:35.200787 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0807 18:18:35.202822 (+ 2035us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:18:35.203830 (+ 1008us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:18:35.203835 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:18:35.204408 (+ 573us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:18:35.204772 (+ 364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:18:35.204790 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:18:35.207175 (+ 2385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:18:35.207179 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:18:35.208274 (+ 1095us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:18:35.208287 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:18:35.209727 (+ 1440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:18:35.209738 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:18:35.209969 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:18:35.210740 (+ 771us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:18:35.210773 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:18:35.213619 (+ 2846us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:18:35.216881 (+ 3262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:18:35.216895 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:18:35.216909 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:18:35.217314 (+ 405us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:18:35.217810 (+ 496us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:18:35.217816 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:18:35.217819 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:18:35.217989 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:18:35.218769 (+ 780us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:18:35.218783 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:18:35.219010 (+ 227us) client_negotiation.cc:769] Sending connection context
0807 18:18:35.219282 (+ 272us) client_negotiation.cc:240] Negotiation successful
0807 18:18:35.219922 (+ 640us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":676,"mutex_wait_us":158,"thread_start_us":422,"threads_started":1}
I20260807 18:18:35.221385 18683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:35.199658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41622 (local address 127.18.154.62:40517)
0807 18:18:35.201425 (+ 1767us) server_negotiation.cc:206] Beginning negotiation
0807 18:18:35.201438 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 18:18:35.201505 (+ 67us) server_negotiation.cc:407] Connection header received
0807 18:18:35.203214 (+ 1709us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:18:35.203224 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:18:35.203555 (+ 331us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:18:35.203788 (+ 233us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:18:35.205003 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:35.207006 (+ 2003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:35.208426 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:35.209406 (+ 980us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:35.209649 (+ 243us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:18:35.213846 (+ 4197us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:18:35.213854 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:18:35.213867 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:18:35.213897 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:18:35.216742 (+ 2845us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:18:35.217458 (+ 716us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:18:35.217464 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:18:35.217467 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:18:35.217694 (+ 227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:18:35.218097 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:18:35.218104 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:18:35.218107 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:18:35.218620 (+ 513us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:18:35.218852 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0807 18:18:35.219399 (+ 547us) server_negotiation.cc:299] Negotiation successful
0807 18:18:35.219809 (+ 410us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":23,"server-negotiator.queue_time_us":1542,"spinlock_wait_cycles":28416,"thread_start_us":1308,"threads_started":1}
I20260807 18:18:35.224694 18678 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.154.61" port: 44067 } has no permanent_uuid. Determining permanent_uuid...
I20260807 18:18:35.225986 18682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:35.225577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.61:44067 (local address 127.0.0.1:49148)
0807 18:18:35.225754 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0807 18:18:35.225871 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.61:44067: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260807 18:18:35.226339 18613 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.154.61:44067: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.18.154.61@KRBTEST.COM; defaulting to no policy
I20260807 18:18:35.228091 18682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:35.227625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.61:44067 (local address 127.0.0.1:49164)
0807 18:18:35.227818 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0807 18:18:35.227920 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.61:44067: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260807 18:18:35.228369 18678 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.154.61:44067: Network error: Client connection negotiation failed: client connection to 127.18.154.61:44067: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.154.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.154.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.61.keytab.
Entry for principal kudu/127.18.154.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.61.keytab.
I20260807 18:18:35.269362 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126328808105-19048-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126328808105-19048-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126328808105-19048-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126328808105-19048-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.154.61:44067
--webserver_interface=127.18.154.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.61.keytab
--principal=kudu/127.18.154.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.154.20:37953
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.154.62:40517,127.18.154.61:44067,127.18.154.60:43209
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126328808105-19048-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126328808105-19048-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126328808105-19048-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 18:18:35.277727 18678 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.154.61" port: 44067 } attempt: 1
I20260807 18:18:35.279192 18682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:35.278586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.61:44067 (local address 127.0.0.1:49174)
0807 18:18:35.278837 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0807 18:18:35.278993 (+ 156us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.61:44067: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260807 18:18:35.281702 18682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:35.280733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.61:44067 (local address 127.0.0.1:49190)
0807 18:18:35.281392 (+ 659us) negotiation.cc:107] Waiting for socket to connect
0807 18:18:35.281492 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.61:44067: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":451}
W20260807 18:18:35.283740 18678 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.154.61:44067: Network error: Client connection negotiation failed: client connection to 127.18.154.61:44067: connect: Connection refused (error 111)
I20260807 18:18:35.352039 18678 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.154.61" port: 44067 } attempt: 2
I20260807 18:18:35.353456 18682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:35.352803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.61:44067 (local address 127.0.0.1:49200)
0807 18:18:35.353128 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0807 18:18:35.353262 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.61:44067: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97}
I20260807 18:18:35.354765 18682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:35.354314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.61:44067 (local address 127.0.0.1:49206)
0807 18:18:35.354478 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0807 18:18:35.354600 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.61:44067: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260807 18:18:35.355053 18678 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.154.61:44067: Network error: Client connection negotiation failed: client connection to 127.18.154.61:44067: connect: Connection refused (error 111)
I20260807 18:18:35.446381 18678 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.154.61" port: 44067 } attempt: 3
I20260807 18:18:35.447827 18682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:35.447179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.61:44067 (local address 127.0.0.1:49208)
0807 18:18:35.447492 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0807 18:18:35.447623 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.61:44067: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134}
I20260807 18:18:35.449002 18682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:35.448603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.61:44067 (local address 127.0.0.1:49218)
0807 18:18:35.448769 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0807 18:18:35.448853 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.61:44067: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260807 18:18:35.449276 18678 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.154.61:44067: Network error: Client connection negotiation failed: client connection to 127.18.154.61:44067: connect: Connection refused (error 111)
I20260807 18:18:35.592618 18678 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.154.61" port: 44067 } attempt: 4
I20260807 18:18:35.593988 18682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:35.593369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.61:44067 (local address 127.0.0.1:49228)
0807 18:18:35.593753 (+ 384us) negotiation.cc:107] Waiting for socket to connect
0807 18:18:35.593843 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.61:44067: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":218}
I20260807 18:18:35.595135 18682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:35.594741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.61:44067 (local address 127.0.0.1:49234)
0807 18:18:35.594906 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0807 18:18:35.594992 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.61:44067: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260807 18:18:35.595364 18678 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.154.61:44067: Network error: Client connection negotiation failed: client connection to 127.18.154.61:44067: connect: Connection refused (error 111)
W20260807 18:18:35.626870 18687 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 18:18:35.627231 18687 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:18:35.627363 18687 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:18:35.627516 18687 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:18:35.634809 18687 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:18:35.634900 18687 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:18:35.634994 18687 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:18:35.635084 18687 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:18:35.635190 18687 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:18:35.645363 18687 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.154.20:37953
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126328808105-19048-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126328808105-19048-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.18.154.62:40517,127.18.154.61:44067,127.18.154.60:43209
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.61.keytab
--principal=kudu/127.18.154.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.154.61:44067
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126328808105-19048-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.18.154.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126328808105-19048-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:18:35.646822 18687 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:18:35.648159 18687 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:18:35.661093 18697 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:18:35.661023 18700 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:18:35.661062 18698 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:18:35.662233 18687 server_base.cc:1034] running on GCE node
I20260807 18:18:35.663012 18687 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:18:35.664445 18687 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:18:35.665666 18687 hybrid_clock.cc:648] HybridClock initialized: now 1786126715665606 us; error 39 us; skew 500 ppm
I20260807 18:18:35.665997 18687 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:18:35 dist-test-slave-gkw7 krb5kdc[18565](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126715, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:18:35.669242 18687 init.cc:377] Logged in from keytab as kudu/127.18.154.61@KRBTEST.COM (short username kudu)
I20260807 18:18:35.670885 18687 webserver.cc:460] Webserver started at http://127.18.154.61:43949/ using document root <none> and password file <none>
I20260807 18:18:35.671608 18687 fs_manager.cc:359] Metadata directory not provided
I20260807 18:18:35.671756 18687 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:18:35.672096 18687 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:18:35.675203 18687 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126328808105-19048-0/minicluster-data/master-1/data/instance:
uuid: "1e0e1d4c408f4e5c83b339a1eeffabee"
format_stamp: "Formatted at 2026-08-07 18:18:35 on dist-test-slave-gkw7"
server_key: "9294229690abbbe08f6b44a01164ee3c"
server_key_iv: "7e69d2aa18547b6b122fa919c364435c"
server_key_version: "encryptionkey@0"
I20260807 18:18:35.676010 18687 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126328808105-19048-0/minicluster-data/master-1/wal/instance:
uuid: "1e0e1d4c408f4e5c83b339a1eeffabee"
format_stamp: "Formatted at 2026-08-07 18:18:35 on dist-test-slave-gkw7"
server_key: "9294229690abbbe08f6b44a01164ee3c"
server_key_iv: "7e69d2aa18547b6b122fa919c364435c"
server_key_version: "encryptionkey@0"
I20260807 18:18:35.681358 18687 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260807 18:18:35.685279 18710 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:18:35.686776 18687 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:18:35.686972 18687 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126328808105-19048-0/minicluster-data/master-1/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126328808105-19048-0/minicluster-data/master-1/wal
uuid: "1e0e1d4c408f4e5c83b339a1eeffabee"
format_stamp: "Formatted at 2026-08-07 18:18:35 on dist-test-slave-gkw7"
server_key: "9294229690abbbe08f6b44a01164ee3c"
server_key_iv: "7e69d2aa18547b6b122fa919c364435c"
server_key_version: "encryptionkey@0"
I20260807 18:18:35.687217 18687 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126328808105-19048-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126328808105-19048-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126328808105-19048-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:18:35.700130 18687 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:18:35.704604 18687 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:18:35.705004 18687 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:18:35.725088 18687 rpc_server.cc:307] RPC server started. Bound to: 127.18.154.61:44067
I20260807 18:18:35.725113 18780 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.154.61:44067 every 8 connection(s)
I20260807 18:18:35.726636 18687 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126328808105-19048-0/minicluster-data/master-1/data/info.pb
I20260807 18:18:35.730611 18781 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:18:35.735011 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 18687
I20260807 18:18:35.735226 19048 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126328808105-19048-0/minicluster-data/master-1/wal/instance
I20260807 18:18:35.735658 19048 external_mini_cluster.cc:1428] Setting key b8be08bcba8191caa5416e8a3b4ec416
I20260807 18:18:35.737468 18781 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.154.62" port: 40517 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 18:18:35 dist-test-slave-gkw7 krb5kdc[18565](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126715, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.61@KRBTEST.COM for kudu/127.18.154.62@KRBTEST.COM
I20260807 18:18:35.758036 18783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:35.740134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41632 (local address 127.18.154.62:40517)
0807 18:18:35.740654 (+ 520us) server_negotiation.cc:206] Beginning negotiation
0807 18:18:35.740659 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:18:35.741692 (+ 1033us) server_negotiation.cc:407] Connection header received
0807 18:18:35.743601 (+ 1909us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:18:35.743609 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:18:35.743705 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:18:35.743830 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:18:35.745403 (+ 1573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:35.746282 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:35.747937 (+ 1655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:35.748349 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:35.748485 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:18:35.752071 (+ 3586us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:18:35.752076 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:18:35.752079 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:18:35.752131 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:18:35.754906 (+ 2775us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:18:35.755655 (+ 749us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:18:35.755662 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:18:35.755667 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:18:35.755790 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:18:35.756217 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:18:35.756222 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:18:35.756226 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:18:35.756509 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:18:35.756689 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0807 18:18:35.757391 (+ 702us) server_negotiation.cc:299] Negotiation successful
0807 18:18:35.757655 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":418,"thread_start_us":292,"threads_started":1}
I20260807 18:18:35.759022 18786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:35.740704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:40517 (local address 127.0.0.1:41632)
0807 18:18:35.741463 (+ 759us) negotiation.cc:107] Waiting for socket to connect
0807 18:18:35.741502 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 18:18:35.743322 (+ 1820us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:18:35.744074 (+ 752us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:18:35.744081 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:18:35.744592 (+ 511us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:18:35.745178 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:18:35.745195 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:18:35.746448 (+ 1253us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:18:35.746452 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:18:35.747779 (+ 1327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:18:35.747792 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:18:35.748591 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:18:35.748609 (+ 18us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:18:35.748822 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:18:35.749518 (+ 696us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:18:35.749591 (+ 73us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:18:35.751842 (+ 2251us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:18:35.755061 (+ 3219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:18:35.755069 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:18:35.755086 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:18:35.755512 (+ 426us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:18:35.755906 (+ 394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:18:35.755911 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:18:35.755915 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:18:35.756102 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:18:35.756651 (+ 549us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:18:35.756658 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:18:35.757007 (+ 349us) client_negotiation.cc:769] Sending connection context
0807 18:18:35.757296 (+ 289us) client_negotiation.cc:240] Negotiation successful
0807 18:18:35.757857 (+ 561us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":526,"thread_start_us":325,"threads_started":1}
I20260807 18:18:35.760882 18781 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.154.61" port: 44067 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.18.154.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.154.60@KRBTEST.COM" created.
Aug 07 18:18:35 dist-test-slave-gkw7 krb5kdc[18565](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126715, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.61@KRBTEST.COM for kudu/127.18.154.61@KRBTEST.COM
I20260807 18:18:35.775895 18786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:35.761727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.61:44067 (local address 127.0.0.1:49242)
0807 18:18:35.761959 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0807 18:18:35.761980 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 18:18:35.762095 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:18:35.763216 (+ 1121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:18:35.763221 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:18:35.763676 (+ 455us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:18:35.763799 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:18:35.763807 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:18:35.765184 (+ 1377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:18:35.765189 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:18:35.766160 (+ 971us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:18:35.766172 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:18:35.766974 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:18:35.766979 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:18:35.767218 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:18:35.767754 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:18:35.767770 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:18:35.770027 (+ 2257us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:18:35.773280 (+ 3253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:18:35.773286 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:18:35.773290 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:18:35.773723 (+ 433us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:18:35.774109 (+ 386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:18:35.774116 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:18:35.774122 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:18:35.774263 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:18:35.774889 (+ 626us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:18:35.774893 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:18:35.775030 (+ 137us) client_negotiation.cc:769] Sending connection context
0807 18:18:35.775177 (+ 147us) client_negotiation.cc:240] Negotiation successful
0807 18:18:35.775466 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":67,"mutex_wait_us":43}
I20260807 18:18:35.775980 18790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:35.762013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49242 (local address 127.18.154.61:44067)
0807 18:18:35.762819 (+ 806us) server_negotiation.cc:206] Beginning negotiation
0807 18:18:35.762826 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:18:35.762851 (+ 25us) server_negotiation.cc:407] Connection header received
0807 18:18:35.762895 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:18:35.762902 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:18:35.763069 (+ 167us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:18:35.763229 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:18:35.763980 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:35.765046 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:35.766290 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:35.766827 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:35.766966 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:18:35.770255 (+ 3289us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:18:35.770268 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:18:35.770282 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:18:35.770309 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:18:35.773094 (+ 2785us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:18:35.773867 (+ 773us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:18:35.773878 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:18:35.773886 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:18:35.773998 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:18:35.774378 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:18:35.774383 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:18:35.774386 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:18:35.774753 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:18:35.774972 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0807 18:18:35.775374 (+ 402us) server_negotiation.cc:299] Negotiation successful
0807 18:18:35.775602 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":656,"thread_start_us":497,"threads_started":1}
I20260807 18:18:35.778625 18781 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.154.60" port: 43209 } has no permanent_uuid. Determining permanent_uuid...
I20260807 18:18:35.779944 18786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:35.779463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.60:43209 (local address 127.0.0.1:58108)
0807 18:18:35.779677 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0807 18:18:35.779781 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.60:43209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260807 18:18:35.780241 18715 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.154.60:43209: connect: Connection refused (error 111)
I20260807 18:18:35.781965 18786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:35.781612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.60:43209 (local address 127.0.0.1:58112)
0807 18:18:35.781756 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0807 18:18:35.781855 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.60:43209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260807 18:18:35.782337 18781 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.154.60:43209: Network error: Client connection negotiation failed: client connection to 127.18.154.60:43209: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.154.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.60.keytab.
Entry for principal kudu/127.18.154.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.60.keytab.
I20260807 18:18:35.808491 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126328808105-19048-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126328808105-19048-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126328808105-19048-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126328808105-19048-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.154.60:43209
--webserver_interface=127.18.154.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.60.keytab
--principal=kudu/127.18.154.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.154.20:37953
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.154.62:40517,127.18.154.61:44067,127.18.154.60:43209
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126328808105-19048-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126328808105-19048-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126328808105-19048-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 18:18:35.831681 18781 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.154.60" port: 43209 } attempt: 1
I20260807 18:18:35.833081 18786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:35.832426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.60:43209 (local address 127.0.0.1:58114)
0807 18:18:35.832712 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0807 18:18:35.832861 (+ 149us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.60:43209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96}
I20260807 18:18:35.834359 18786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:35.833915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.60:43209 (local address 127.0.0.1:58128)
0807 18:18:35.834145 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0807 18:18:35.834232 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.60:43209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
W20260807 18:18:35.834671 18781 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.154.60:43209: Network error: Client connection negotiation failed: client connection to 127.18.154.60:43209: connect: Connection refused (error 111)
I20260807 18:18:35.894656 18678 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.154.61" port: 44067 } attempt: 5
I20260807 18:18:35.902912 18781 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.154.60" port: 43209 } attempt: 2
Aug 07 18:18:35 dist-test-slave-gkw7 krb5kdc[18565](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126715, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.62@KRBTEST.COM for kudu/127.18.154.61@KRBTEST.COM
I20260807 18:18:35.903861 18786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:35.903444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.60:43209 (local address 127.0.0.1:58136)
0807 18:18:35.903623 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0807 18:18:35.903710 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.60:43209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260807 18:18:35.904920 18786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:35.904532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.60:43209 (local address 127.0.0.1:58150)
0807 18:18:35.904684 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0807 18:18:35.904771 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.60:43209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260807 18:18:35.905339 18781 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.154.60:43209: Network error: Client connection negotiation failed: client connection to 127.18.154.60:43209: connect: Connection refused (error 111)
I20260807 18:18:35.910454 18790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:35.895689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49256 (local address 127.18.154.61:44067)
0807 18:18:35.895935 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0807 18:18:35.895943 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:18:35.896015 (+ 72us) server_negotiation.cc:407] Connection header received
0807 18:18:35.896094 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:18:35.896103 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:18:35.896214 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:18:35.896345 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:18:35.897196 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:35.898531 (+ 1335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:35.899734 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:35.900388 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:35.900522 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:18:35.904140 (+ 3618us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:18:35.904147 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:18:35.904152 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:18:35.904191 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:18:35.907688 (+ 3497us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:18:35.908374 (+ 686us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:18:35.908380 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:18:35.908384 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:18:35.908542 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:18:35.908908 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:18:35.908913 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:18:35.908916 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:18:35.909296 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:18:35.909438 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0807 18:18:35.909947 (+ 509us) server_negotiation.cc:299] Negotiation successful
0807 18:18:35.910158 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260807 18:18:35.910501 18682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:35.895575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.61:44067 (local address 127.0.0.1:49256)
0807 18:18:35.895802 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0807 18:18:35.895826 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 18:18:35.895933 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:18:35.896384 (+ 451us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:18:35.896388 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:18:35.896816 (+ 428us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:18:35.897005 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:18:35.897014 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:18:35.898682 (+ 1668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:18:35.898686 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:18:35.899564 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:18:35.899576 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:18:35.900542 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:18:35.900547 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:18:35.900722 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:18:35.901415 (+ 693us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:18:35.901432 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:18:35.903904 (+ 2472us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:18:35.907835 (+ 3931us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:18:35.907839 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:18:35.907842 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:18:35.908227 (+ 385us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:18:35.908661 (+ 434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:18:35.908664 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:18:35.908666 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:18:35.908800 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:18:35.909415 (+ 615us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:18:35.909418 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:18:35.909640 (+ 222us) client_negotiation.cc:769] Sending connection context
0807 18:18:35.909837 (+ 197us) client_negotiation.cc:240] Negotiation successful
0807 18:18:35.910132 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":70}
I20260807 18:18:35.911680 18678 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.154.60" port: 43209 } has no permanent_uuid. Determining permanent_uuid...
I20260807 18:18:35.912845 18682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:35.912466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.60:43209 (local address 127.0.0.1:58166)
0807 18:18:35.912635 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0807 18:18:35.912710 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.60:43209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260807 18:18:35.913826 18682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:35.913463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.60:43209 (local address 127.0.0.1:58170)
0807 18:18:35.913636 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0807 18:18:35.913718 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.60:43209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260807 18:18:35.914043 18678 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.154.60:43209: Network error: Client connection negotiation failed: client connection to 127.18.154.60:43209: connect: Connection refused (error 111)
I20260807 18:18:35.965319 18678 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.154.60" port: 43209 } attempt: 1
I20260807 18:18:35.966678 18682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:35.966112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.60:43209 (local address 127.0.0.1:58182)
0807 18:18:35.966373 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0807 18:18:35.966473 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.60:43209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260807 18:18:35.967885 18682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:35.967510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.60:43209 (local address 127.0.0.1:58198)
0807 18:18:35.967689 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0807 18:18:35.967776 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.60:43209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
W20260807 18:18:35.968144 18678 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.154.60:43209: Network error: Client connection negotiation failed: client connection to 127.18.154.60:43209: connect: Connection refused (error 111)
I20260807 18:18:35.996578 18781 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.154.60" port: 43209 } attempt: 3
I20260807 18:18:35.997885 18786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:35.997293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.60:43209 (local address 127.0.0.1:58208)
0807 18:18:35.997550 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0807 18:18:35.997685 (+ 135us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.60:43209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260807 18:18:35.999123 18786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:35.998641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.60:43209 (local address 127.0.0.1:58224)
0807 18:18:35.998889 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0807 18:18:35.998987 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.60:43209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144}
W20260807 18:18:35.999392 18781 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.154.60:43209: Network error: Client connection negotiation failed: client connection to 127.18.154.60:43209: connect: Connection refused (error 111)
I20260807 18:18:36.036427 18678 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.154.60" port: 43209 } attempt: 2
I20260807 18:18:36.037899 18682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:36.037194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.60:43209 (local address 127.0.0.1:58230)
0807 18:18:36.037487 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0807 18:18:36.037654 (+ 167us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.60:43209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98}
I20260807 18:18:36.039186 18682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:36.038719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.60:43209 (local address 127.0.0.1:58246)
0807 18:18:36.038924 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0807 18:18:36.039026 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.60:43209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260807 18:18:36.039472 18678 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.154.60:43209: Network error: Client connection negotiation failed: client connection to 127.18.154.60:43209: connect: Connection refused (error 111)
I20260807 18:18:36.142673 18781 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.154.60" port: 43209 } attempt: 4
I20260807 18:18:36.144029 18786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:36.143476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.60:43209 (local address 127.0.0.1:58260)
0807 18:18:36.143762 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0807 18:18:36.143858 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.60:43209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106}
I20260807 18:18:36.145176 18786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:36.144834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.60:43209 (local address 127.0.0.1:58270)
0807 18:18:36.144977 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0807 18:18:36.145062 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.60:43209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260807 18:18:36.145434 18781 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.154.60:43209: Network error: Client connection negotiation failed: client connection to 127.18.154.60:43209: connect: Connection refused (error 111)
I20260807 18:18:36.145691 18678 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.154.60" port: 43209 } attempt: 3
I20260807 18:18:36.146773 18682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:36.146292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.60:43209 (local address 127.0.0.1:58272)
0807 18:18:36.146484 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0807 18:18:36.146583 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.60:43209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260807 18:18:36.147956 18682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:36.147474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.60:43209 (local address 127.0.0.1:58280)
0807 18:18:36.147708 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0807 18:18:36.147806 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.60:43209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95}
W20260807 18:18:36.148227 18678 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.154.60:43209: Network error: Client connection negotiation failed: client connection to 127.18.154.60:43209: connect: Connection refused (error 111)
W20260807 18:18:36.174074 18793 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 18:18:36.174433 18793 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:18:36.174542 18793 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:18:36.174669 18793 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:18:36.182288 18793 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:18:36.182379 18793 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:18:36.182477 18793 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:18:36.182546 18793 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:18:36.182683 18793 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:18:36.193465 18793 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.154.20:37953
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126328808105-19048-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126328808105-19048-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.18.154.62:40517,127.18.154.61:44067,127.18.154.60:43209
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.60.keytab
--principal=kudu/127.18.154.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.154.60:43209
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126328808105-19048-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.18.154.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126328808105-19048-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:18:36.194882 18793 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:18:36.196238 18793 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:18:36.209049 18803 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:18:36.209177 18806 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:18:36.209142 18802 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:18:36.209951 18793 server_base.cc:1034] running on GCE node
I20260807 18:18:36.210713 18793 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:18:36.212148 18793 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:18:36.213414 18793 hybrid_clock.cc:648] HybridClock initialized: now 1786126716213320 us; error 60 us; skew 500 ppm
I20260807 18:18:36.213853 18793 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:18:36 dist-test-slave-gkw7 krb5kdc[18565](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126716, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:18:36.217236 18793 init.cc:377] Logged in from keytab as kudu/127.18.154.60@KRBTEST.COM (short username kudu)
I20260807 18:18:36.218847 18793 webserver.cc:460] Webserver started at http://127.18.154.60:38481/ using document root <none> and password file <none>
I20260807 18:18:36.219513 18793 fs_manager.cc:359] Metadata directory not provided
I20260807 18:18:36.219604 18793 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:18:36.219888 18793 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:18:36.223014 18793 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126328808105-19048-0/minicluster-data/master-2/data/instance:
uuid: "57865456baec4f46a859e3bd7ab9f7c3"
format_stamp: "Formatted at 2026-08-07 18:18:36 on dist-test-slave-gkw7"
server_key: "8b19bc048350be804908b41f6edcd8a0"
server_key_iv: "9c844fc55bfa4ea76b1697a1baf009f8"
server_key_version: "encryptionkey@0"
I20260807 18:18:36.223789 18793 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126328808105-19048-0/minicluster-data/master-2/wal/instance:
uuid: "57865456baec4f46a859e3bd7ab9f7c3"
format_stamp: "Formatted at 2026-08-07 18:18:36 on dist-test-slave-gkw7"
server_key: "8b19bc048350be804908b41f6edcd8a0"
server_key_iv: "9c844fc55bfa4ea76b1697a1baf009f8"
server_key_version: "encryptionkey@0"
I20260807 18:18:36.229080 18793 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260807 18:18:36.232672 18816 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:18:36.234299 18793 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:18:36.234503 18793 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126328808105-19048-0/minicluster-data/master-2/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126328808105-19048-0/minicluster-data/master-2/wal
uuid: "57865456baec4f46a859e3bd7ab9f7c3"
format_stamp: "Formatted at 2026-08-07 18:18:36 on dist-test-slave-gkw7"
server_key: "8b19bc048350be804908b41f6edcd8a0"
server_key_iv: "9c844fc55bfa4ea76b1697a1baf009f8"
server_key_version: "encryptionkey@0"
I20260807 18:18:36.234786 18793 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126328808105-19048-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126328808105-19048-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126328808105-19048-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:18:36.252355 18793 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:18:36.256747 18793 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:18:36.257056 18793 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:18:36.276695 18793 rpc_server.cc:307] RPC server started. Bound to: 127.18.154.60:43209
I20260807 18:18:36.276717 18894 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.154.60:43209 every 8 connection(s)
I20260807 18:18:36.278167 18793 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126328808105-19048-0/minicluster-data/master-2/data/info.pb
I20260807 18:18:36.282198 18895 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:18:36.283063 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 18793
I20260807 18:18:36.283273 19048 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786126328808105-19048-0/minicluster-data/master-2/wal/instance
I20260807 18:18:36.283849 19048 external_mini_cluster.cc:1428] Setting key a133962ea97a94aa63229e3544f6f28a
I20260807 18:18:36.291743 18895 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.154.62" port: 40517 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 18:18:36 dist-test-slave-gkw7 krb5kdc[18565](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126714, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.154.60@KRBTEST.COM
Aug 07 18:18:36 dist-test-slave-gkw7 krb5kdc[18565](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126716, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.60@KRBTEST.COM for kudu/127.18.154.62@KRBTEST.COM
I20260807 18:18:36.309733 18905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:36.285944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45246 (local address 127.18.154.60:43209)
0807 18:18:36.286728 (+ 784us) server_negotiation.cc:206] Beginning negotiation
0807 18:18:36.286740 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 18:18:36.286786 (+ 46us) server_negotiation.cc:407] Connection header received
0807 18:18:36.288792 (+ 2006us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:18:36.288818 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:18:36.289243 (+ 425us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:18:36.289688 (+ 445us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:18:36.291371 (+ 1683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:36.292909 (+ 1538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:36.294832 (+ 1923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:36.295727 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:36.295916 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:18:36.301112 (+ 5196us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:18:36.301125 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:18:36.301148 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:18:36.301196 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:18:36.304739 (+ 3543us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:18:36.305491 (+ 752us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:18:36.305501 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:18:36.305514 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:18:36.305748 (+ 234us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:18:36.306278 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:18:36.306284 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:18:36.306287 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:18:36.307089 (+ 802us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:18:36.307398 (+ 309us) server_negotiation.cc:1035] Waiting for connection context
0807 18:18:36.307823 (+ 425us) server_negotiation.cc:299] Negotiation successful
0807 18:18:36.308287 (+ 464us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":581,"thread_start_us":362,"threads_started":1}
I20260807 18:18:36.311472 18907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:36.294443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:40517 (local address 127.0.0.1:48030)
0807 18:18:36.295273 (+ 830us) negotiation.cc:107] Waiting for socket to connect
0807 18:18:36.295321 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 18:18:36.295488 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:18:36.295934 (+ 446us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:18:36.295941 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:18:36.296836 (+ 895us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:18:36.297219 (+ 383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:18:36.297235 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:18:36.298431 (+ 1196us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:18:36.298437 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:18:36.299829 (+ 1392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:18:36.299844 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:18:36.301004 (+ 1160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:18:36.301017 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:18:36.301213 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:18:36.302001 (+ 788us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:18:36.302020 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:18:36.304872 (+ 2852us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:18:36.308783 (+ 3911us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:18:36.308793 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:18:36.308799 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:18:36.309246 (+ 447us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:18:36.309671 (+ 425us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:18:36.309678 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:18:36.309680 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:18:36.309837 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:18:36.310476 (+ 639us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:18:36.310483 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:18:36.310680 (+ 197us) client_negotiation.cc:769] Sending connection context
0807 18:18:36.310885 (+ 205us) client_negotiation.cc:240] Negotiation successful
0807 18:18:36.311172 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":572,"thread_start_us":417,"threads_started":1}
I20260807 18:18:36.311637 18906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:36.294418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48030 (local address 127.18.154.62:40517)
0807 18:18:36.294968 (+ 550us) server_negotiation.cc:206] Beginning negotiation
0807 18:18:36.294974 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:18:36.295434 (+ 460us) server_negotiation.cc:407] Connection header received
0807 18:18:36.295625 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:18:36.295631 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:18:36.295741 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:18:36.295854 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:18:36.297426 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:36.298254 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:36.300008 (+ 1754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:36.300720 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:36.300881 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:18:36.305114 (+ 4233us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:18:36.305120 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:18:36.305123 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:18:36.305158 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:18:36.308639 (+ 3481us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:18:36.309369 (+ 730us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:18:36.309375 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:18:36.309379 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:18:36.309562 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:18:36.310007 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:18:36.310012 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:18:36.310015 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:18:36.310336 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:18:36.310442 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0807 18:18:36.310961 (+ 519us) server_negotiation.cc:299] Negotiation successful
0807 18:18:36.311198 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":431,"thread_start_us":279,"threads_started":1}
I20260807 18:18:36.313009 18895 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.154.61" port: 44067 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 18:18:36 dist-test-slave-gkw7 krb5kdc[18565](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126716, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.60@KRBTEST.COM for kudu/127.18.154.61@KRBTEST.COM
Aug 07 18:18:36 dist-test-slave-gkw7 krb5kdc[18565](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126714, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.154.62@KRBTEST.COM
I20260807 18:18:36.325484 18678 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.154.60" port: 43209 } attempt: 4
I20260807 18:18:36.330171 18907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:36.313840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.61:44067 (local address 127.0.0.1:47792)
0807 18:18:36.314131 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0807 18:18:36.314149 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 18:18:36.314260 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:18:36.314574 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:18:36.314579 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:18:36.315064 (+ 485us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:18:36.315260 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:18:36.315272 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:18:36.316701 (+ 1429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:18:36.316707 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:18:36.318194 (+ 1487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:18:36.318209 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:18:36.319159 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:18:36.319164 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:18:36.319360 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:18:36.320035 (+ 675us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:18:36.320063 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:18:36.322836 (+ 2773us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:18:36.327312 (+ 4476us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:18:36.327318 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:18:36.327323 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:18:36.327871 (+ 548us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:18:36.328299 (+ 428us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:18:36.328304 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:18:36.328308 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:18:36.328508 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:18:36.329152 (+ 644us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:18:36.329157 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:18:36.329317 (+ 160us) client_negotiation.cc:769] Sending connection context
0807 18:18:36.329456 (+ 139us) client_negotiation.cc:240] Negotiation successful
0807 18:18:36.329779 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168}
I20260807 18:18:36.330214 18790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:36.313795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47792 (local address 127.18.154.61:44067)
0807 18:18:36.313978 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0807 18:18:36.313982 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:18:36.314219 (+ 237us) server_negotiation.cc:407] Connection header received
0807 18:18:36.314371 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:18:36.314375 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:18:36.314433 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:18:36.314510 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:18:36.315416 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:36.316525 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:36.318351 (+ 1826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:36.318971 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:36.319086 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:18:36.323048 (+ 3962us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:18:36.323053 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:18:36.323057 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:18:36.323093 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:18:36.327157 (+ 4064us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:18:36.327996 (+ 839us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:18:36.328002 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:18:36.328005 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:18:36.328172 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:18:36.328656 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:18:36.328663 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:18:36.328667 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:18:36.329026 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:18:36.329186 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0807 18:18:36.329600 (+ 414us) server_negotiation.cc:299] Negotiation successful
0807 18:18:36.329814 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 18:18:36.331513 18906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:36.313207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48042 (local address 127.18.154.62:40517)
0807 18:18:36.313425 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0807 18:18:36.313431 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:18:36.313476 (+ 45us) server_negotiation.cc:407] Connection header received
0807 18:18:36.313559 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:18:36.313567 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:18:36.313663 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:18:36.313762 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:18:36.315522 (+ 1760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:36.316556 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:36.318350 (+ 1794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:36.318958 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:36.319086 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:18:36.324349 (+ 5263us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:18:36.324354 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:18:36.324357 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:18:36.324388 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:18:36.328063 (+ 3675us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:18:36.328877 (+ 814us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:18:36.328883 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:18:36.328887 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:18:36.329048 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:18:36.329582 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:18:36.329587 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:18:36.329591 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:18:36.330012 (+ 421us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:18:36.330343 (+ 331us) server_negotiation.cc:1035] Waiting for connection context
0807 18:18:36.330955 (+ 612us) server_negotiation.cc:299] Negotiation successful
0807 18:18:36.331178 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101}
I20260807 18:18:36.331815 18895 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.154.60" port: 43209 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 18:18:36 dist-test-slave-gkw7 krb5kdc[18565](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126715, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.62@KRBTEST.COM for kudu/127.18.154.60@KRBTEST.COM
I20260807 18:18:36.343791 18682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:36.326230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.60:43209 (local address 127.0.0.1:45250)
0807 18:18:36.326408 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0807 18:18:36.326430 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 18:18:36.326533 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:18:36.326879 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:18:36.326884 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:18:36.327404 (+ 520us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:18:36.327625 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:18:36.327639 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:18:36.329225 (+ 1586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:18:36.329232 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:18:36.331002 (+ 1770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:18:36.331019 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:18:36.332511 (+ 1492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:18:36.332518 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:18:36.332771 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:18:36.333395 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:18:36.333420 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:18:36.336334 (+ 2914us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:18:36.340608 (+ 4274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:18:36.340615 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:18:36.340619 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:18:36.341035 (+ 416us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:18:36.341456 (+ 421us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:18:36.341463 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:18:36.341467 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:18:36.341798 (+ 331us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:18:36.342583 (+ 785us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:18:36.342591 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:18:36.342766 (+ 175us) client_negotiation.cc:769] Sending connection context
0807 18:18:36.342955 (+ 189us) client_negotiation.cc:240] Negotiation successful
0807 18:18:36.343296 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56}
I20260807 18:18:36.343816 18905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:36.326294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45250 (local address 127.18.154.60:43209)
0807 18:18:36.326523 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0807 18:18:36.326529 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:18:36.326547 (+ 18us) server_negotiation.cc:407] Connection header received
0807 18:18:36.326651 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:18:36.326656 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:18:36.326727 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:18:36.326824 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:18:36.327804 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:36.329030 (+ 1226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:36.331224 (+ 2194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:36.332335 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:36.332497 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:18:36.336545 (+ 4048us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:18:36.336559 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:18:36.336563 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:18:36.336595 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:18:36.340440 (+ 3845us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:18:36.341174 (+ 734us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:18:36.341181 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:18:36.341185 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:18:36.341324 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:18:36.341966 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:18:36.341971 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:18:36.341974 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:18:36.342424 (+ 450us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:18:36.342576 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0807 18:18:36.343086 (+ 510us) server_negotiation.cc:299] Negotiation successful
0807 18:18:36.343358 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117}
Aug 07 18:18:36 dist-test-slave-gkw7 krb5kdc[18565](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126714, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.154.61@KRBTEST.COM
Aug 07 18:18:36 dist-test-slave-gkw7 krb5kdc[18565](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126716, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.60@KRBTEST.COM for kudu/127.18.154.60@KRBTEST.COM
I20260807 18:18:36.350589 18678 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 80ebcd8bbf9746a18b1dd35d401b7769: Bootstrap starting.
I20260807 18:18:36.354615 18790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:36.333768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47798 (local address 127.18.154.61:44067)
0807 18:18:36.333975 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0807 18:18:36.333983 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:18:36.334005 (+ 22us) server_negotiation.cc:407] Connection header received
0807 18:18:36.334069 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:18:36.334077 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:18:36.334164 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:18:36.334295 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:18:36.336182 (+ 1887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:36.337653 (+ 1471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:36.340080 (+ 2427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:36.341022 (+ 942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:36.341181 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:18:36.347808 (+ 6627us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:18:36.347822 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:18:36.347825 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:18:36.347861 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:18:36.351534 (+ 3673us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:18:36.352280 (+ 746us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:18:36.352287 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:18:36.352294 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:18:36.352486 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:18:36.352975 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:18:36.352982 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:18:36.352986 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:18:36.353336 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:18:36.353664 (+ 328us) server_negotiation.cc:1035] Waiting for connection context
0807 18:18:36.354031 (+ 367us) server_negotiation.cc:299] Negotiation successful
0807 18:18:36.354274 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 18:18:36.354866 18678 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 80ebcd8bbf9746a18b1dd35d401b7769: Neither blocks nor log segments found. Creating new log.
I20260807 18:18:36.356249 18678 log.cc:824] T 00000000000000000000000000000000 P 80ebcd8bbf9746a18b1dd35d401b7769: Log is configured to *not* fsync() on all Append() calls
I20260807 18:18:36.358217 18907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:36.332792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.60:43209 (local address 127.0.0.1:45264)
0807 18:18:36.333144 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0807 18:18:36.333165 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 18:18:36.333274 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:18:36.342494 (+ 9220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:18:36.342503 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:18:36.342933 (+ 430us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:18:36.343152 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:18:36.343165 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:18:36.344890 (+ 1725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:18:36.344899 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:18:36.346542 (+ 1643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:18:36.346560 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:18:36.347546 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:18:36.347552 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:18:36.347764 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:18:36.348403 (+ 639us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:18:36.348429 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:18:36.351177 (+ 2748us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:18:36.355515 (+ 4338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:18:36.355519 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:18:36.355521 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:18:36.355961 (+ 440us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:18:36.356395 (+ 434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:18:36.356400 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:18:36.356403 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:18:36.356572 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:18:36.357208 (+ 636us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:18:36.357213 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:18:36.357391 (+ 178us) client_negotiation.cc:769] Sending connection context
0807 18:18:36.357641 (+ 250us) client_negotiation.cc:240] Negotiation successful
0807 18:18:36.357947 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":218}
I20260807 18:18:36.358626 18908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:36.333382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45264 (local address 127.18.154.60:43209)
0807 18:18:36.342155 (+ 8773us) server_negotiation.cc:206] Beginning negotiation
0807 18:18:36.342161 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:18:36.342188 (+ 27us) server_negotiation.cc:407] Connection header received
0807 18:18:36.342239 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:18:36.342244 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:18:36.342317 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:18:36.343298 (+ 981us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:18:36.343558 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:36.344755 (+ 1197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:36.346731 (+ 1976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:36.347391 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:36.348042 (+ 651us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:18:36.351415 (+ 3373us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:18:36.351422 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:18:36.351426 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:18:36.351461 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:18:36.354993 (+ 3532us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:18:36.356082 (+ 1089us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:18:36.356088 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:18:36.356092 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:18:36.356249 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:18:36.356688 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:18:36.356693 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:18:36.356695 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:18:36.357064 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:18:36.357238 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0807 18:18:36.357957 (+ 719us) server_negotiation.cc:299] Negotiation successful
0807 18:18:36.358219 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8584,"thread_start_us":8351,"threads_started":1}
I20260807 18:18:36.360293 19048 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:18:36.360736 18678 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 80ebcd8bbf9746a18b1dd35d401b7769: No bootstrap required, opened a new log
I20260807 18:18:36.364761 18895 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 57865456baec4f46a859e3bd7ab9f7c3: Bootstrap starting.
I20260807 18:18:36.363340 18647 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "80ebcd8bbf9746a18b1dd35d401b7769"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48042
I20260807 18:18:36.368822 18895 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 57865456baec4f46a859e3bd7ab9f7c3: Neither blocks nor log segments found. Creating new log.
I20260807 18:18:36.370189 18895 log.cc:824] T 00000000000000000000000000000000 P 57865456baec4f46a859e3bd7ab9f7c3: Log is configured to *not* fsync() on all Append() calls
I20260807 18:18:36.371881 18678 raft_consensus.cc:348] T 00000000000000000000000000000000 P 80ebcd8bbf9746a18b1dd35d401b7769 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80ebcd8bbf9746a18b1dd35d401b7769" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 40517 } } peers { permanent_uuid: "1e0e1d4c408f4e5c83b339a1eeffabee" member_type: VOTER last_known_addr { host: "127.18.154.61" port: 44067 } } peers { permanent_uuid: "57865456baec4f46a859e3bd7ab9f7c3" member_type: VOTER last_known_addr { host: "127.18.154.60" port: 43209 } }
I20260807 18:18:36.372272 18678 raft_consensus.cc:729] T 00000000000000000000000000000000 P 80ebcd8bbf9746a18b1dd35d401b7769 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80ebcd8bbf9746a18b1dd35d401b7769, State: Initialized, Role: FOLLOWER
I20260807 18:18:36.372823 18678 consensus_queue.cc:260] T 00000000000000000000000000000000 P 80ebcd8bbf9746a18b1dd35d401b7769 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80ebcd8bbf9746a18b1dd35d401b7769" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 40517 } } peers { permanent_uuid: "1e0e1d4c408f4e5c83b339a1eeffabee" member_type: VOTER last_known_addr { host: "127.18.154.61" port: 44067 } } peers { permanent_uuid: "57865456baec4f46a859e3bd7ab9f7c3" member_type: VOTER last_known_addr { host: "127.18.154.60" port: 43209 } }
I20260807 18:18:36.373811 18895 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 57865456baec4f46a859e3bd7ab9f7c3: No bootstrap required, opened a new log
I20260807 18:18:36.373878 18912 sys_catalog.cc:455] T 00000000000000000000000000000000 P 80ebcd8bbf9746a18b1dd35d401b7769 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80ebcd8bbf9746a18b1dd35d401b7769" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 40517 } } peers { permanent_uuid: "1e0e1d4c408f4e5c83b339a1eeffabee" member_type: VOTER last_known_addr { host: "127.18.154.61" port: 44067 } } peers { permanent_uuid: "57865456baec4f46a859e3bd7ab9f7c3" member_type: VOTER last_known_addr { host: "127.18.154.60" port: 43209 } } }
I20260807 18:18:36.374307 18912 sys_catalog.cc:458] T 00000000000000000000000000000000 P 80ebcd8bbf9746a18b1dd35d401b7769 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 18:18:36.375352 18678 sys_catalog.cc:564] T 00000000000000000000000000000000 P 80ebcd8bbf9746a18b1dd35d401b7769 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:18:36.375422 18647 raft_consensus.cc:482] T 00000000000000000000000000000000 P 80ebcd8bbf9746a18b1dd35d401b7769 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 18:18:36.375677 18647 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 80ebcd8bbf9746a18b1dd35d401b7769 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:18:36.378765 18647 raft_consensus.cc:504] T 00000000000000000000000000000000 P 80ebcd8bbf9746a18b1dd35d401b7769 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80ebcd8bbf9746a18b1dd35d401b7769" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 40517 } } peers { permanent_uuid: "1e0e1d4c408f4e5c83b339a1eeffabee" member_type: VOTER last_known_addr { host: "127.18.154.61" port: 44067 } } peers { permanent_uuid: "57865456baec4f46a859e3bd7ab9f7c3" member_type: VOTER last_known_addr { host: "127.18.154.60" port: 43209 } }
I20260807 18:18:36.381325 18647 leader_election.cc:288] T 00000000000000000000000000000000 P 80ebcd8bbf9746a18b1dd35d401b7769 [CANDIDATE]: Term 1 election: Requested vote from peers 1e0e1d4c408f4e5c83b339a1eeffabee (127.18.154.61:44067), 57865456baec4f46a859e3bd7ab9f7c3 (127.18.154.60:43209)
I20260807 18:18:36.388334 18895 raft_consensus.cc:348] T 00000000000000000000000000000000 P 57865456baec4f46a859e3bd7ab9f7c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80ebcd8bbf9746a18b1dd35d401b7769" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 40517 } } peers { permanent_uuid: "1e0e1d4c408f4e5c83b339a1eeffabee" member_type: VOTER last_known_addr { host: "127.18.154.61" port: 44067 } } peers { permanent_uuid: "57865456baec4f46a859e3bd7ab9f7c3" member_type: VOTER last_known_addr { host: "127.18.154.60" port: 43209 } }
I20260807 18:18:36.388955 18895 raft_consensus.cc:729] T 00000000000000000000000000000000 P 57865456baec4f46a859e3bd7ab9f7c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57865456baec4f46a859e3bd7ab9f7c3, State: Initialized, Role: FOLLOWER
I20260807 18:18:36.389700 18895 consensus_queue.cc:260] T 00000000000000000000000000000000 P 57865456baec4f46a859e3bd7ab9f7c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80ebcd8bbf9746a18b1dd35d401b7769" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 40517 } } peers { permanent_uuid: "1e0e1d4c408f4e5c83b339a1eeffabee" member_type: VOTER last_known_addr { host: "127.18.154.61" port: 44067 } } peers { permanent_uuid: "57865456baec4f46a859e3bd7ab9f7c3" member_type: VOTER last_known_addr { host: "127.18.154.60" port: 43209 } }
I20260807 18:18:36.393658 18856 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "80ebcd8bbf9746a18b1dd35d401b7769" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "57865456baec4f46a859e3bd7ab9f7c3"
I20260807 18:18:36.393544 18895 sys_catalog.cc:564] T 00000000000000000000000000000000 P 57865456baec4f46a859e3bd7ab9f7c3 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:18:36.394305 18856 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 57865456baec4f46a859e3bd7ab9f7c3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:18:36.391219 18930 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57865456baec4f46a859e3bd7ab9f7c3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80ebcd8bbf9746a18b1dd35d401b7769" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 40517 } } peers { permanent_uuid: "1e0e1d4c408f4e5c83b339a1eeffabee" member_type: VOTER last_known_addr { host: "127.18.154.61" port: 44067 } } peers { permanent_uuid: "57865456baec4f46a859e3bd7ab9f7c3" member_type: VOTER last_known_addr { host: "127.18.154.60" port: 43209 } } }
I20260807 18:18:36.395985 18930 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57865456baec4f46a859e3bd7ab9f7c3 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 18:18:36.400035 18856 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 57865456baec4f46a859e3bd7ab9f7c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80ebcd8bbf9746a18b1dd35d401b7769 in term 1.
I20260807 18:18:36.401198 18616 leader_election.cc:302] T 00000000000000000000000000000000 P 80ebcd8bbf9746a18b1dd35d401b7769 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57865456baec4f46a859e3bd7ab9f7c3, 80ebcd8bbf9746a18b1dd35d401b7769; no voters:
I20260807 18:18:36.401852 18912 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 80ebcd8bbf9746a18b1dd35d401b7769 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:18:36.405319 18912 raft_consensus.cc:686] T 00000000000000000000000000000000 P 80ebcd8bbf9746a18b1dd35d401b7769 [term 1 LEADER]: Becoming Leader. State: Replica: 80ebcd8bbf9746a18b1dd35d401b7769, State: Running, Role: LEADER
I20260807 18:18:36.406054 18912 consensus_queue.cc:237] T 00000000000000000000000000000000 P 80ebcd8bbf9746a18b1dd35d401b7769 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80ebcd8bbf9746a18b1dd35d401b7769" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 40517 } } peers { permanent_uuid: "1e0e1d4c408f4e5c83b339a1eeffabee" member_type: VOTER last_known_addr { host: "127.18.154.61" port: 44067 } } peers { permanent_uuid: "57865456baec4f46a859e3bd7ab9f7c3" member_type: VOTER last_known_addr { host: "127.18.154.60" port: 43209 } }
I20260807 18:18:36.412878 18938 sys_catalog.cc:455] T 00000000000000000000000000000000 P 80ebcd8bbf9746a18b1dd35d401b7769 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 80ebcd8bbf9746a18b1dd35d401b7769. Latest consensus state: current_term: 1 leader_uuid: "80ebcd8bbf9746a18b1dd35d401b7769" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80ebcd8bbf9746a18b1dd35d401b7769" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 40517 } } peers { permanent_uuid: "1e0e1d4c408f4e5c83b339a1eeffabee" member_type: VOTER last_known_addr { host: "127.18.154.61" port: 44067 } } peers { permanent_uuid: "57865456baec4f46a859e3bd7ab9f7c3" member_type: VOTER last_known_addr { host: "127.18.154.60" port: 43209 } } }
I20260807 18:18:36.413443 18938 sys_catalog.cc:458] T 00000000000000000000000000000000 P 80ebcd8bbf9746a18b1dd35d401b7769 [sys.catalog]: This master's current role is: LEADER
W20260807 18:18:36.417621 18619 tablet.cc:2369] T 00000000000000000000000000000000 P 80ebcd8bbf9746a18b1dd35d401b7769: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:18:36.434887 18946 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:18:36.441661 18906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:36.385843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48046 (local address 127.18.154.62:40517)
0807 18:18:36.386138 (+ 295us) server_negotiation.cc:206] Beginning negotiation
0807 18:18:36.386146 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:18:36.400134 (+ 13988us) server_negotiation.cc:407] Connection header received
0807 18:18:36.401434 (+ 1300us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:18:36.401440 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:18:36.401552 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:18:36.401629 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:18:36.410355 (+ 8726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:36.412188 (+ 1833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:36.418514 (+ 6326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:36.422462 (+ 3948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:36.422597 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:18:36.428094 (+ 5497us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:18:36.428102 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:18:36.428109 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:18:36.428153 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:18:36.432225 (+ 4072us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:18:36.436348 (+ 4123us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:18:36.436353 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:18:36.436382 (+ 29us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:18:36.436674 (+ 292us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:18:36.437595 (+ 921us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:18:36.437602 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:18:36.437606 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:18:36.438562 (+ 956us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:18:36.439148 (+ 586us) server_negotiation.cc:1035] Waiting for connection context
0807 18:18:36.440399 (+ 1251us) server_negotiation.cc:299] Negotiation successful
0807 18:18:36.440871 (+ 472us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146}
I20260807 18:18:36.444670 18781 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.154.60" port: 43209 } attempt: 5
I20260807 18:18:36.446827 18790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:36.386301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47814 (local address 127.18.154.61:44067)
0807 18:18:36.386525 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0807 18:18:36.386533 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:18:36.406620 (+ 20087us) server_negotiation.cc:407] Connection header received
0807 18:18:36.407230 (+ 610us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:18:36.407238 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:18:36.407352 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:18:36.407438 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:18:36.411907 (+ 4469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:36.416521 (+ 4614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:36.420670 (+ 4149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:36.421344 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:36.421463 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:18:36.437254 (+ 15791us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:18:36.437261 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:18:36.437266 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:18:36.437315 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:18:36.441254 (+ 3939us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:18:36.442313 (+ 1059us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:18:36.442320 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:18:36.442324 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:18:36.442506 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:18:36.442975 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:18:36.442981 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:18:36.442984 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:18:36.445264 (+ 2280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:18:36.445510 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0807 18:18:36.446166 (+ 656us) server_negotiation.cc:299] Negotiation successful
0807 18:18:36.446417 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260807 18:18:36.447453 18908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:36.392165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45278 (local address 127.18.154.60:43209)
0807 18:18:36.392374 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0807 18:18:36.392383 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:18:36.411687 (+ 19304us) server_negotiation.cc:407] Connection header received
0807 18:18:36.411882 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:18:36.411887 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:18:36.411994 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:18:36.412093 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:18:36.416019 (+ 3926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:36.417041 (+ 1022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:36.420196 (+ 3155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:36.420894 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:36.420994 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:18:36.430982 (+ 9988us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:18:36.430993 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:18:36.430999 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:18:36.431051 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:18:36.434951 (+ 3900us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:18:36.444133 (+ 9182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:18:36.444139 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:18:36.444142 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:18:36.444323 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:18:36.445105 (+ 782us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:18:36.445110 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:18:36.445113 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:18:36.445582 (+ 469us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:18:36.445915 (+ 333us) server_negotiation.cc:1035] Waiting for connection context
0807 18:18:36.446772 (+ 857us) server_negotiation.cc:299] Negotiation successful
0807 18:18:36.447123 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 18:18:36.454721 18946 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 07 18:18:36 dist-test-slave-gkw7 krb5kdc[18565](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126715, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.61@KRBTEST.COM for kudu/127.18.154.60@KRBTEST.COM
I20260807 18:18:36.470685 18905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:36.446294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45286 (local address 127.18.154.60:43209)
0807 18:18:36.446446 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0807 18:18:36.446453 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:18:36.446474 (+ 21us) server_negotiation.cc:407] Connection header received
0807 18:18:36.446561 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:18:36.446569 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:18:36.446662 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:18:36.446792 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:18:36.448185 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:36.449650 (+ 1465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:36.452510 (+ 2860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:36.453443 (+ 933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:36.453607 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:18:36.459538 (+ 5931us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:18:36.459546 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:18:36.459553 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:18:36.459596 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:18:36.464054 (+ 4458us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:18:36.465336 (+ 1282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:18:36.465345 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:18:36.465382 (+ 37us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:18:36.465643 (+ 261us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:18:36.466389 (+ 746us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:18:36.466397 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:18:36.466400 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:18:36.467157 (+ 757us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:18:36.467420 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0807 18:18:36.469640 (+ 2220us) server_negotiation.cc:299] Negotiation successful
0807 18:18:36.470044 (+ 404us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260807 18:18:36.472326 18786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:36.445315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.60:43209 (local address 127.0.0.1:45286)
0807 18:18:36.445858 (+ 543us) negotiation.cc:107] Waiting for socket to connect
0807 18:18:36.445882 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 18:18:36.446010 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:18:36.447013 (+ 1003us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:18:36.447021 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:18:36.447688 (+ 667us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:18:36.447950 (+ 262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:18:36.447966 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:18:36.449885 (+ 1919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:18:36.449894 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:18:36.451818 (+ 1924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:18:36.451838 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:18:36.453884 (+ 2046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:18:36.453893 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:18:36.454222 (+ 329us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:18:36.455040 (+ 818us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:18:36.455072 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:18:36.459261 (+ 4189us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:18:36.464267 (+ 5006us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:18:36.464275 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:18:36.464279 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:18:36.465122 (+ 843us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:18:36.465894 (+ 772us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:18:36.465898 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:18:36.465902 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:18:36.466177 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:18:36.467564 (+ 1387us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:18:36.467571 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:18:36.467833 (+ 262us) client_negotiation.cc:769] Sending connection context
0807 18:18:36.471252 (+ 3419us) client_negotiation.cc:240] Negotiation successful
0807 18:18:36.471738 (+ 486us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":361}
I20260807 18:18:36.477478 18967 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260807 18:18:36.479305 18781 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e0e1d4c408f4e5c83b339a1eeffabee: Bootstrap starting.
I20260807 18:18:36.481423 18856 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 57865456baec4f46a859e3bd7ab9f7c3 [term 1 FOLLOWER]: Refusing update from remote peer 80ebcd8bbf9746a18b1dd35d401b7769: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:18:36.482880 18938 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 80ebcd8bbf9746a18b1dd35d401b7769 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57865456baec4f46a859e3bd7ab9f7c3" member_type: VOTER last_known_addr { host: "127.18.154.60" port: 43209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:18:36.483955 18781 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1e0e1d4c408f4e5c83b339a1eeffabee: Neither blocks nor log segments found. Creating new log.
I20260807 18:18:36.486289 18781 log.cc:824] T 00000000000000000000000000000000 P 1e0e1d4c408f4e5c83b339a1eeffabee: Log is configured to *not* fsync() on all Append() calls
I20260807 18:18:36.488437 18963 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260807 18:18:36.496769 18781 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e0e1d4c408f4e5c83b339a1eeffabee: No bootstrap required, opened a new log
I20260807 18:18:36.492267 18589 master_runner.cc:186] Error getting master registration for 127.18.154.61:44067: OK, instance_id { permanent_uuid: "1e0e1d4c408f4e5c83b339a1eeffabee" instance_seqno: 1786126715719456 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 18:18:36.503473 18589 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 18:18:36.480229 18793 master_runner.cc:186] Error getting master registration for 127.18.154.62:40517: OK, instance_id { permanent_uuid: "80ebcd8bbf9746a18b1dd35d401b7769" instance_seqno: 1786126715178090 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 18:18:36.505894 18793 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 18:18:36.507293 18930 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57865456baec4f46a859e3bd7ab9f7c3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 80ebcd8bbf9746a18b1dd35d401b7769. Latest consensus state: current_term: 1 leader_uuid: "80ebcd8bbf9746a18b1dd35d401b7769" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80ebcd8bbf9746a18b1dd35d401b7769" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 40517 } } peers { permanent_uuid: "1e0e1d4c408f4e5c83b339a1eeffabee" member_type: VOTER last_known_addr { host: "127.18.154.61" port: 44067 } } peers { permanent_uuid: "57865456baec4f46a859e3bd7ab9f7c3" member_type: VOTER last_known_addr { host: "127.18.154.60" port: 43209 } } }
I20260807 18:18:36.507608 18930 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57865456baec4f46a859e3bd7ab9f7c3 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 18:18:36.508692 18781 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1e0e1d4c408f4e5c83b339a1eeffabee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80ebcd8bbf9746a18b1dd35d401b7769" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 40517 } } peers { permanent_uuid: "1e0e1d4c408f4e5c83b339a1eeffabee" member_type: VOTER last_known_addr { host: "127.18.154.61" port: 44067 } } peers { permanent_uuid: "57865456baec4f46a859e3bd7ab9f7c3" member_type: VOTER last_known_addr { host: "127.18.154.60" port: 43209 } }
I20260807 18:18:36.509148 18781 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1e0e1d4c408f4e5c83b339a1eeffabee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e0e1d4c408f4e5c83b339a1eeffabee, State: Initialized, Role: FOLLOWER
W20260807 18:18:36.509337 18824 tablet.cc:2369] T 00000000000000000000000000000000 P 57865456baec4f46a859e3bd7ab9f7c3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:18:36.509855 18781 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1e0e1d4c408f4e5c83b339a1eeffabee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80ebcd8bbf9746a18b1dd35d401b7769" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 40517 } } peers { permanent_uuid: "1e0e1d4c408f4e5c83b339a1eeffabee" member_type: VOTER last_known_addr { host: "127.18.154.61" port: 44067 } } peers { permanent_uuid: "57865456baec4f46a859e3bd7ab9f7c3" member_type: VOTER last_known_addr { host: "127.18.154.60" port: 43209 } }
I20260807 18:18:36.511972 18961 mvcc.cc:205] Tried to move back new op lower bound from 7315975030683058176 to 7315975030406524928. Current Snapshot: MvccSnapshot[applied={T|T < 7315975030683058176}]
I20260807 18:18:36.512311 18781 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1e0e1d4c408f4e5c83b339a1eeffabee [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:18:36.510912 18971 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e0e1d4c408f4e5c83b339a1eeffabee [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80ebcd8bbf9746a18b1dd35d401b7769" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 40517 } } peers { permanent_uuid: "1e0e1d4c408f4e5c83b339a1eeffabee" member_type: VOTER last_known_addr { host: "127.18.154.61" port: 44067 } } peers { permanent_uuid: "57865456baec4f46a859e3bd7ab9f7c3" member_type: VOTER last_known_addr { host: "127.18.154.60" port: 43209 } } }
I20260807 18:18:36.512528 18743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "80ebcd8bbf9746a18b1dd35d401b7769" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "1e0e1d4c408f4e5c83b339a1eeffabee"
I20260807 18:18:36.512857 18971 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e0e1d4c408f4e5c83b339a1eeffabee [sys.catalog]: This master's current role is: FOLLOWER
I20260807 18:18:36.513482 18742 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1e0e1d4c408f4e5c83b339a1eeffabee [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:18:36.513384 18930 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57865456baec4f46a859e3bd7ab9f7c3 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "80ebcd8bbf9746a18b1dd35d401b7769" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80ebcd8bbf9746a18b1dd35d401b7769" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 40517 } } peers { permanent_uuid: "1e0e1d4c408f4e5c83b339a1eeffabee" member_type: VOTER last_known_addr { host: "127.18.154.61" port: 44067 } } peers { permanent_uuid: "57865456baec4f46a859e3bd7ab9f7c3" member_type: VOTER last_known_addr { host: "127.18.154.60" port: 43209 } } }
I20260807 18:18:36.513657 18930 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57865456baec4f46a859e3bd7ab9f7c3 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 18:18:36.517207 18742 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1e0e1d4c408f4e5c83b339a1eeffabee [term 1 FOLLOWER]: Refusing update from remote peer 80ebcd8bbf9746a18b1dd35d401b7769: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:18:36.519866 18743 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1e0e1d4c408f4e5c83b339a1eeffabee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80ebcd8bbf9746a18b1dd35d401b7769 in term 1.
I20260807 18:18:36.520444 18938 sys_catalog.cc:455] T 00000000000000000000000000000000 P 80ebcd8bbf9746a18b1dd35d401b7769 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "80ebcd8bbf9746a18b1dd35d401b7769" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80ebcd8bbf9746a18b1dd35d401b7769" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 40517 } } peers { permanent_uuid: "1e0e1d4c408f4e5c83b339a1eeffabee" member_type: VOTER last_known_addr { host: "127.18.154.61" port: 44067 } } peers { permanent_uuid: "57865456baec4f46a859e3bd7ab9f7c3" member_type: VOTER last_known_addr { host: "127.18.154.60" port: 43209 } } }
I20260807 18:18:36.520777 18938 sys_catalog.cc:458] T 00000000000000000000000000000000 P 80ebcd8bbf9746a18b1dd35d401b7769 [sys.catalog]: This master's current role is: LEADER
I20260807 18:18:36.520974 18938 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 80ebcd8bbf9746a18b1dd35d401b7769 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1e0e1d4c408f4e5c83b339a1eeffabee" member_type: VOTER last_known_addr { host: "127.18.154.61" port: 44067 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:18:36.524761 18946 catalog_manager.cc:1347] Generated new cluster ID: e44fed9bbeb441efa6a2743f14b0e6df
I20260807 18:18:36.524863 18946 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:18:36.551669 18971 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e0e1d4c408f4e5c83b339a1eeffabee [sys.catalog]: SysCatalogTable state changed. Reason: New leader 80ebcd8bbf9746a18b1dd35d401b7769. Latest consensus state: current_term: 1 leader_uuid: "80ebcd8bbf9746a18b1dd35d401b7769" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80ebcd8bbf9746a18b1dd35d401b7769" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 40517 } } peers { permanent_uuid: "1e0e1d4c408f4e5c83b339a1eeffabee" member_type: VOTER last_known_addr { host: "127.18.154.61" port: 44067 } } peers { permanent_uuid: "57865456baec4f46a859e3bd7ab9f7c3" member_type: VOTER last_known_addr { host: "127.18.154.60" port: 43209 } } }
I20260807 18:18:36.551980 18971 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e0e1d4c408f4e5c83b339a1eeffabee [sys.catalog]: This master's current role is: FOLLOWER
I20260807 18:18:36.552495 18986 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e0e1d4c408f4e5c83b339a1eeffabee [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "80ebcd8bbf9746a18b1dd35d401b7769" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80ebcd8bbf9746a18b1dd35d401b7769" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 40517 } } peers { permanent_uuid: "1e0e1d4c408f4e5c83b339a1eeffabee" member_type: VOTER last_known_addr { host: "127.18.154.61" port: 44067 } } peers { permanent_uuid: "57865456baec4f46a859e3bd7ab9f7c3" member_type: VOTER last_known_addr { host: "127.18.154.60" port: 43209 } } }
I20260807 18:18:36.552918 18986 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e0e1d4c408f4e5c83b339a1eeffabee [sys.catalog]: This master's current role is: FOLLOWER
I20260807 18:18:36.558648 18938 sys_catalog.cc:455] T 00000000000000000000000000000000 P 80ebcd8bbf9746a18b1dd35d401b7769 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "80ebcd8bbf9746a18b1dd35d401b7769" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80ebcd8bbf9746a18b1dd35d401b7769" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 40517 } } peers { permanent_uuid: "1e0e1d4c408f4e5c83b339a1eeffabee" member_type: VOTER last_known_addr { host: "127.18.154.61" port: 44067 } } peers { permanent_uuid: "57865456baec4f46a859e3bd7ab9f7c3" member_type: VOTER last_known_addr { host: "127.18.154.60" port: 43209 } } }
I20260807 18:18:36.558913 18938 sys_catalog.cc:458] T 00000000000000000000000000000000 P 80ebcd8bbf9746a18b1dd35d401b7769 [sys.catalog]: This master's current role is: LEADER
I20260807 18:18:36.563375 18946 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:18:36.565322 18946 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:18:36.584787 19003 catalog_manager.cc:1259] Loaded cluster ID: e44fed9bbeb441efa6a2743f14b0e6df
I20260807 18:18:36.584983 19003 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 1e0e1d4c408f4e5c83b339a1eeffabee: loading cluster ID for follower catalog manager: success
I20260807 18:18:36.588564 19003 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 1e0e1d4c408f4e5c83b339a1eeffabee: acquiring CA information for follower catalog manager: success
W20260807 18:18:36.589897 19003 catalog_manager.cc:873] Not found: no TSK found in the system table: failed to prepare follower catalog manager, will retry
I20260807 18:18:36.600952 18946 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 80ebcd8bbf9746a18b1dd35d401b7769: Generated new TSK 0
I20260807 18:18:36.602296 18946 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 18:18:36.720754 18790 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:18:36.696444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47826 (local address 127.18.154.61:44067)
0807 18:18:36.696702 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0807 18:18:36.696709 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:18:36.718682 (+ 21973us) server_negotiation.cc:407] Connection header received
0807 18:18:36.719018 (+ 336us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:18:36.719025 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:18:36.719114 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:18:36.719195 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:18:36.720464 (+ 1269us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47826: BlockingRecv error: recv got EOF from 127.0.0.1:47826 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
W20260807 18:18:36.723476 18906 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:18:36.704823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48048 (local address 127.18.154.62:40517)
0807 18:18:36.705099 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0807 18:18:36.705107 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:18:36.721673 (+ 16566us) server_negotiation.cc:407] Connection header received
0807 18:18:36.721991 (+ 318us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:18:36.721999 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:18:36.722086 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:18:36.722158 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:18:36.723211 (+ 1053us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48048: BlockingRecv error: recv got EOF from 127.0.0.1:48048 (error 108)
Metrics: {"server-negotiator.queue_time_us":87}
W20260807 18:18:36.726419 18905 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:18:36.711456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45300 (local address 127.18.154.60:43209)
0807 18:18:36.711897 (+ 441us) server_negotiation.cc:206] Beginning negotiation
0807 18:18:36.711906 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:18:36.724747 (+ 12841us) server_negotiation.cc:407] Connection header received
0807 18:18:36.725007 (+ 260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:18:36.725013 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:18:36.725109 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:18:36.725181 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:18:36.726164 (+ 983us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45300: BlockingRecv error: recv got EOF from 127.0.0.1:45300 (error 108)
Metrics: {"server-negotiator.queue_time_us":192}
W20260807 18:18:36.776281 18790 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:18:36.767216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47834 (local address 127.18.154.61:44067)
0807 18:18:36.767438 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0807 18:18:36.767446 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:18:36.774244 (+ 6798us) server_negotiation.cc:407] Connection header received
0807 18:18:36.774566 (+ 322us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:18:36.774583 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:18:36.774725 (+ 142us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:18:36.774834 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:18:36.775947 (+ 1113us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47834: BlockingRecv error: recv got EOF from 127.0.0.1:47834 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
W20260807 18:18:36.811590 18790 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:18:36.802266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47840 (local address 127.18.154.61:44067)
0807 18:18:36.802473 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0807 18:18:36.802480 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:18:36.809503 (+ 7023us) server_negotiation.cc:407] Connection header received
0807 18:18:36.810084 (+ 581us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:18:36.810091 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:18:36.810192 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:18:36.810281 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:18:36.811342 (+ 1061us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47840: BlockingRecv error: recv got EOF from 127.0.0.1:47840 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
W20260807 18:18:36.822053 18790 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:18:36.819482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47842 (local address 127.18.154.61:44067)
0807 18:18:36.819686 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0807 18:18:36.819692 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:18:36.820187 (+ 495us) server_negotiation.cc:407] Connection header received
0807 18:18:36.820493 (+ 306us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:18:36.820498 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:18:36.820621 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:18:36.820709 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:18:36.821793 (+ 1084us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47842: BlockingRecv error: recv got EOF from 127.0.0.1:47842 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
W20260807 18:18:36.832947 18790 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:18:36.830356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47848 (local address 127.18.154.61:44067)
0807 18:18:36.830608 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0807 18:18:36.830615 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:18:36.831151 (+ 536us) server_negotiation.cc:407] Connection header received
0807 18:18:36.831455 (+ 304us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:18:36.831461 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:18:36.831552 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:18:36.831647 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:18:36.832677 (+ 1030us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47848: BlockingRecv error: recv got EOF from 127.0.0.1:47848 (error 108)
Metrics: {"server-negotiator.queue_time_us":121}
W20260807 18:18:36.848858 18790 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:18:36.845993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47862 (local address 127.18.154.61:44067)
0807 18:18:36.846188 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0807 18:18:36.846194 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:18:36.847177 (+ 983us) server_negotiation.cc:407] Connection header received
0807 18:18:36.847392 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:18:36.847398 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:18:36.847491 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:18:36.847616 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:18:36.848600 (+ 984us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47862: BlockingRecv error: recv got EOF from 127.0.0.1:47862 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
W20260807 18:18:36.873014 18790 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:18:36.870021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47872 (local address 127.18.154.61:44067)
0807 18:18:36.870212 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0807 18:18:36.870219 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:18:36.870779 (+ 560us) server_negotiation.cc:407] Connection header received
0807 18:18:36.871141 (+ 362us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:18:36.871149 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:18:36.871275 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:18:36.871364 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:18:36.872745 (+ 1381us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47872: BlockingRecv error: recv got EOF from 127.0.0.1:47872 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
W20260807 18:18:36.914427 18790 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:18:36.911471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47880 (local address 127.18.154.61:44067)
0807 18:18:36.911664 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0807 18:18:36.911671 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:18:36.912210 (+ 539us) server_negotiation.cc:407] Connection header received
0807 18:18:36.912960 (+ 750us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:18:36.912966 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:18:36.913055 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:18:36.913126 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:18:36.914161 (+ 1035us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47880: BlockingRecv error: recv got EOF from 127.0.0.1:47880 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
W20260807 18:18:36.998677 18790 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:18:36.995789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47884 (local address 127.18.154.61:44067)
0807 18:18:36.996035 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0807 18:18:36.996042 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:18:36.996601 (+ 559us) server_negotiation.cc:407] Connection header received
0807 18:18:36.996958 (+ 357us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:18:36.996965 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:18:36.997079 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:18:36.997162 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:18:36.998387 (+ 1225us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47884: BlockingRecv error: recv got EOF from 127.0.0.1:47884 (error 108)
Metrics: {"server-negotiator.queue_time_us":90}
W20260807 18:18:37.137406 18790 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:18:37.135348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47894 (local address 127.18.154.61:44067)
0807 18:18:37.135529 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0807 18:18:37.135534 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:18:37.135885 (+ 351us) server_negotiation.cc:407] Connection header received
0807 18:18:37.136087 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:18:37.136093 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:18:37.136180 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:18:37.136282 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:18:37.137125 (+ 843us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47894: BlockingRecv error: recv got EOF from 127.0.0.1:47894 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
W20260807 18:18:37.399734 18790 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:18:37.397680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47896 (local address 127.18.154.61:44067)
0807 18:18:37.397951 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0807 18:18:37.397959 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:18:37.398188 (+ 229us) server_negotiation.cc:407] Connection header received
0807 18:18:37.398457 (+ 269us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:18:37.398462 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:18:37.398536 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:18:37.398657 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:18:37.399544 (+ 887us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47896: BlockingRecv error: recv got EOF from 127.0.0.1:47896 (error 108)
Metrics: {"server-negotiator.queue_time_us":108}
I20260807 18:18:37.482626 18967 catalog_manager.cc:1259] Loaded cluster ID: e44fed9bbeb441efa6a2743f14b0e6df
I20260807 18:18:37.482790 18967 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 57865456baec4f46a859e3bd7ab9f7c3: loading cluster ID for follower catalog manager: success
I20260807 18:18:37.485618 18967 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 57865456baec4f46a859e3bd7ab9f7c3: acquiring CA information for follower catalog manager: success
I20260807 18:18:37.487562 18967 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 57865456baec4f46a859e3bd7ab9f7c3: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 18:18:37.593952 19003 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 1e0e1d4c408f4e5c83b339a1eeffabee: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 18:18:37.922071 19104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:37.915898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47906 (local address 127.18.154.61:44067)
0807 18:18:37.916412 (+ 514us) server_negotiation.cc:206] Beginning negotiation
0807 18:18:37.916418 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:18:37.916580 (+ 162us) server_negotiation.cc:407] Connection header received
0807 18:18:37.916829 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:18:37.916833 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:18:37.916992 (+ 159us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:18:37.917130 (+ 138us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:18:37.917623 (+ 493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:37.918335 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:37.919692 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:37.920094 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:37.920229 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:18:37.920754 (+ 525us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:18:37.921419 (+ 665us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:18:37.921628 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0807 18:18:37.921681 (+ 53us) server_negotiation.cc:299] Negotiation successful
0807 18:18:37.921762 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":381,"thread_start_us":244,"threads_started":1}
I20260807 18:18:37.933955 19114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:37.928101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45312 (local address 127.18.154.60:43209)
0807 18:18:37.928673 (+ 572us) server_negotiation.cc:206] Beginning negotiation
0807 18:18:37.928678 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:18:37.928695 (+ 17us) server_negotiation.cc:407] Connection header received
0807 18:18:37.928813 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:18:37.928817 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:18:37.928971 (+ 154us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:18:37.929139 (+ 168us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:18:37.929485 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:37.930340 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:37.931664 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:37.932077 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:37.932180 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:18:37.932705 (+ 525us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:18:37.933274 (+ 569us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:18:37.933396 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0807 18:18:37.933608 (+ 212us) server_negotiation.cc:299] Negotiation successful
0807 18:18:37.933682 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":464,"thread_start_us":314,"threads_started":1}
I20260807 18:18:37.959293 19104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:37.940907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47918 (local address 127.18.154.61:44067)
0807 18:18:37.941143 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0807 18:18:37.941152 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:18:37.952398 (+ 11246us) server_negotiation.cc:407] Connection header received
0807 18:18:37.952626 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:18:37.952635 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:18:37.952744 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:18:37.952847 (+ 103us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:18:37.954085 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:37.954843 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:37.957073 (+ 2230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:37.957735 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:37.957862 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:18:37.958503 (+ 641us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:18:37.958653 (+ 150us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:18:37.958759 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0807 18:18:37.958882 (+ 123us) server_negotiation.cc:299] Negotiation successful
0807 18:18:37.958964 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260807 18:18:37.959605 19126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:37.940716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48052 (local address 127.18.154.62:40517)
0807 18:18:37.951647 (+ 10931us) server_negotiation.cc:206] Beginning negotiation
0807 18:18:37.951658 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 18:18:37.952300 (+ 642us) server_negotiation.cc:407] Connection header received
0807 18:18:37.952565 (+ 265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:18:37.952575 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:18:37.952852 (+ 277us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:18:37.953051 (+ 199us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:18:37.953727 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:37.954847 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:37.956855 (+ 2008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:37.957490 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:37.957732 (+ 242us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:18:37.958471 (+ 739us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:18:37.958977 (+ 506us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:18:37.959106 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0807 18:18:37.959210 (+ 104us) server_negotiation.cc:299] Negotiation successful
0807 18:18:37.959296 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10729,"thread_start_us":10580,"threads_started":1}
I20260807 18:18:37.960393 19114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:37.941046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45320 (local address 127.18.154.60:43209)
0807 18:18:37.945734 (+ 4688us) server_negotiation.cc:206] Beginning negotiation
0807 18:18:37.945743 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:18:37.953118 (+ 7375us) server_negotiation.cc:407] Connection header received
0807 18:18:37.953281 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:18:37.953285 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:18:37.953352 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:18:37.953460 (+ 108us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:18:37.954284 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:37.955357 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:37.957467 (+ 2110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:37.958067 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:37.958207 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:18:37.958670 (+ 463us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:18:37.958780 (+ 110us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:18:37.958871 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0807 18:18:37.960050 (+ 1179us) server_negotiation.cc:299] Negotiation successful
0807 18:18:37.960144 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4517}
I20260807 18:18:37.968835 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 18589
I20260807 18:18:37.969974 18673 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:18:38.082275 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 18589
I20260807 18:18:38.113329 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 18687
I20260807 18:18:38.114619 18772 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:18:38.224015 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 18687
I20260807 18:18:38.254081 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 18793
I20260807 18:18:38.255365 18888 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:18:38.363051 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 18793
2026-08-07T18:18:38Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (5328 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126328808105-19048-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:18:38 dist-test-slave-gkw7 krb5kdc[19132](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:18:38 dist-test-slave-gkw7 krb5kdc[19132](info): set up 2 sockets
Aug 07 18:18:38 dist-test-slave-gkw7 krb5kdc[19132](info): commencing operation
krb5kdc: starting...
W20260807 18:18:39.714813 19048 mini_kdc.cc:121] Time spent starting KDC: real 1.304s user 0.004s sys 0.047s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126328808105-19048-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126328808105-19048-0/krb5kdc/test-user.keytab.
Aug 07 18:18:39 dist-test-slave-gkw7 krb5kdc[19132](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126719, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:18:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:18:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.154.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.154.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.154.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.62.keytab.
Entry for principal kudu/127.18.154.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.62.keytab.
I20260807 18:18:40.065802 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126328808105-19048-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126328808105-19048-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126328808105-19048-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.154.62:33241
--webserver_interface=127.18.154.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.62.keytab
--principal=kudu/127.18.154.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.154.20:35749
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.154.62:33241,127.18.154.61:32933,127.18.154.60:36053
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126328808105-19048-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126328808105-19048-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126328808105-19048-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:18:40.431660 19150 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 18:18:40.432031 19150 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:18:40.432170 19150 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:18:40.432304 19150 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:18:40.439826 19150 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:18:40.439924 19150 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:18:40.440023 19150 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:18:40.440121 19150 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:18:40.440237 19150 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:18:40.450505 19150 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.154.20:35749
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126328808105-19048-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126328808105-19048-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.154.62:33241,127.18.154.61:32933,127.18.154.60:36053
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.62.keytab
--principal=kudu/127.18.154.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.154.62:33241
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.154.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126328808105-19048-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:18:40.451880 19150 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:18:40.453272 19150 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:18:40.465435 19160 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:18:40.465952 19161 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:18:40.465958 19164 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:18:40.466898 19150 server_base.cc:1034] running on GCE node
I20260807 18:18:40.467730 19150 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:18:40.469147 19150 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:18:40.470366 19150 hybrid_clock.cc:648] HybridClock initialized: now 1786126720470310 us; error 29 us; skew 500 ppm
I20260807 18:18:40.470732 19150 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:18:40 dist-test-slave-gkw7 krb5kdc[19132](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126720, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:18:40.474087 19150 init.cc:377] Logged in from keytab as kudu/127.18.154.62@KRBTEST.COM (short username kudu)
I20260807 18:18:40.475625 19150 webserver.cc:460] Webserver started at http://127.18.154.62:46231/ using document root <none> and password file <none>
I20260807 18:18:40.476351 19150 fs_manager.cc:359] Metadata directory not provided
I20260807 18:18:40.476444 19150 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:18:40.476854 19150 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:18:40.480041 19150 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126328808105-19048-0/minicluster-data/master-0/data/instance:
uuid: "e82bc516d09f4f9aa49c6d18eb877f01"
format_stamp: "Formatted at 2026-08-07 18:18:40 on dist-test-slave-gkw7"
server_key: "dc5653fa921ac8d4b60531d8ce662ac3"
server_key_iv: "7ab2e8f348d198add059dadf05e004e9"
server_key_version: "encryptionkey@0"
I20260807 18:18:40.480882 19150 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126328808105-19048-0/minicluster-data/master-0/wal/instance:
uuid: "e82bc516d09f4f9aa49c6d18eb877f01"
format_stamp: "Formatted at 2026-08-07 18:18:40 on dist-test-slave-gkw7"
server_key: "dc5653fa921ac8d4b60531d8ce662ac3"
server_key_iv: "7ab2e8f348d198add059dadf05e004e9"
server_key_version: "encryptionkey@0"
I20260807 18:18:40.486465 19150 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260807 18:18:40.490159 19173 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:18:40.491573 19150 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260807 18:18:40.491792 19150 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126328808105-19048-0/minicluster-data/master-0/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126328808105-19048-0/minicluster-data/master-0/wal
uuid: "e82bc516d09f4f9aa49c6d18eb877f01"
format_stamp: "Formatted at 2026-08-07 18:18:40 on dist-test-slave-gkw7"
server_key: "dc5653fa921ac8d4b60531d8ce662ac3"
server_key_iv: "7ab2e8f348d198add059dadf05e004e9"
server_key_version: "encryptionkey@0"
I20260807 18:18:40.492015 19150 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126328808105-19048-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126328808105-19048-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126328808105-19048-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:18:40.506563 19150 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:18:40.511474 19150 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:18:40.511781 19150 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:18:40.531737 19150 rpc_server.cc:307] RPC server started. Bound to: 127.18.154.62:33241
I20260807 18:18:40.531744 19244 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.154.62:33241 every 8 connection(s)
I20260807 18:18:40.533223 19150 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126328808105-19048-0/minicluster-data/master-0/data/info.pb
I20260807 18:18:40.537299 19245 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:18:40.540499 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 19150
I20260807 18:18:40.540719 19048 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126328808105-19048-0/minicluster-data/master-0/wal/instance
I20260807 18:18:40.541273 19048 external_mini_cluster.cc:1428] Setting key f67c79d0b830e2fe9c2f1bf2e44c00e9
I20260807 18:18:40.546861 19245 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.154.62" port: 33241 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 18:18:40 dist-test-slave-gkw7 krb5kdc[19132](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126720, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.62@KRBTEST.COM for kudu/127.18.154.62@KRBTEST.COM
I20260807 18:18:40.574805 19249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:40.551104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:33241 (local address 127.0.0.1:42750)
0807 18:18:40.552099 (+ 995us) negotiation.cc:107] Waiting for socket to connect
0807 18:18:40.552158 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0807 18:18:40.555179 (+ 3021us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:18:40.556151 (+ 972us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:18:40.556161 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:18:40.556848 (+ 687us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:18:40.557200 (+ 352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:18:40.557218 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:18:40.559670 (+ 2452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:18:40.559674 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:18:40.560831 (+ 1157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:18:40.560844 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:18:40.562295 (+ 1451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:18:40.562301 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:18:40.562439 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:18:40.563179 (+ 740us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:18:40.563206 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:18:40.565458 (+ 2252us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:18:40.570224 (+ 4766us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:18:40.570235 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:18:40.570245 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:18:40.570684 (+ 439us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:18:40.571215 (+ 531us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:18:40.571219 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:18:40.571222 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:18:40.571445 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:18:40.572527 (+ 1082us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:18:40.572534 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:18:40.572729 (+ 195us) client_negotiation.cc:769] Sending connection context
0807 18:18:40.573061 (+ 332us) client_negotiation.cc:240] Negotiation successful
0807 18:18:40.573618 (+ 557us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":658,"thread_start_us":402,"threads_started":1}
I20260807 18:18:40.574847 19250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:40.551114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42750 (local address 127.18.154.62:33241)
0807 18:18:40.553123 (+ 2009us) server_negotiation.cc:206] Beginning negotiation
0807 18:18:40.553129 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:18:40.553170 (+ 41us) server_negotiation.cc:407] Connection header received
0807 18:18:40.555620 (+ 2450us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:18:40.555626 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:18:40.555878 (+ 252us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:18:40.556052 (+ 174us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:18:40.557432 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:40.559485 (+ 2053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:40.560987 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:40.562056 (+ 1069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:40.562271 (+ 215us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:18:40.565737 (+ 3466us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:18:40.565750 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:18:40.565769 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:18:40.565811 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:18:40.570045 (+ 4234us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:18:40.570832 (+ 787us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:18:40.570845 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:18:40.570851 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:18:40.571050 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:18:40.571582 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:18:40.571590 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:18:40.571596 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:18:40.572368 (+ 772us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:18:40.572582 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0807 18:18:40.573259 (+ 677us) server_negotiation.cc:299] Negotiation successful
0807 18:18:40.573784 (+ 525us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":175,"server-negotiator.queue_time_us":1812,"spinlock_wait_cycles":28544,"thread_start_us":1565,"threads_started":1}
WARNING: no policy specified for kudu/127.18.154.61@KRBTEST.COM; defaulting to no policy
I20260807 18:18:40.578434 19245 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.154.61" port: 32933 } has no permanent_uuid. Determining permanent_uuid...
I20260807 18:18:40.579577 19249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:40.579118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.61:32933 (local address 127.0.0.1:36758)
0807 18:18:40.579332 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0807 18:18:40.579445 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.61:32933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100}
W20260807 18:18:40.579888 19177 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.154.61:32933: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.154.61@KRBTEST.COM" created.
I20260807 18:18:40.581665 19249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:40.581160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.61:32933 (local address 127.0.0.1:36762)
0807 18:18:40.581334 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0807 18:18:40.581443 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.61:32933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260807 18:18:40.581914 19245 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.154.61:32933: Network error: Client connection negotiation failed: client connection to 127.18.154.61:32933: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.154.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.61.keytab.
Entry for principal kudu/127.18.154.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.61.keytab.
I20260807 18:18:40.620805 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126328808105-19048-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126328808105-19048-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126328808105-19048-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126328808105-19048-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.154.61:32933
--webserver_interface=127.18.154.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.61.keytab
--principal=kudu/127.18.154.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.154.20:35749
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.154.62:33241,127.18.154.61:32933,127.18.154.60:36053
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126328808105-19048-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126328808105-19048-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126328808105-19048-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 18:18:40.631312 19245 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.154.61" port: 32933 } attempt: 1
I20260807 18:18:40.632606 19249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:40.632110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.61:32933 (local address 127.0.0.1:36774)
0807 18:18:40.632347 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0807 18:18:40.632460 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.61:32933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99}
I20260807 18:18:40.633790 19249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:40.633309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.61:32933 (local address 127.0.0.1:36782)
0807 18:18:40.633483 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0807 18:18:40.633597 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.61:32933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260807 18:18:40.634151 19245 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.154.61:32933: Network error: Client connection negotiation failed: client connection to 127.18.154.61:32933: connect: Connection refused (error 111)
I20260807 18:18:40.702473 19245 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.154.61" port: 32933 } attempt: 2
I20260807 18:18:40.703886 19249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:40.703219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.61:32933 (local address 127.0.0.1:36788)
0807 18:18:40.703524 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0807 18:18:40.703688 (+ 164us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.61:32933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94}
I20260807 18:18:40.705173 19249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:40.704703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.61:32933 (local address 127.0.0.1:36804)
0807 18:18:40.704885 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0807 18:18:40.704994 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.61:32933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260807 18:18:40.705416 19245 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.154.61:32933: Network error: Client connection negotiation failed: client connection to 127.18.154.61:32933: connect: Connection refused (error 111)
I20260807 18:18:40.796748 19245 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.154.61" port: 32933 } attempt: 3
I20260807 18:18:40.798272 19249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:40.797609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.61:32933 (local address 127.0.0.1:36816)
0807 18:18:40.797875 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0807 18:18:40.798012 (+ 137us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.61:32933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260807 18:18:40.799746 19249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:40.799213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.61:32933 (local address 127.0.0.1:36828)
0807 18:18:40.799419 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0807 18:18:40.799523 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.61:32933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
W20260807 18:18:40.800061 19245 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.154.61:32933: Network error: Client connection negotiation failed: client connection to 127.18.154.61:32933: connect: Connection refused (error 111)
I20260807 18:18:40.943387 19245 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.154.61" port: 32933 } attempt: 4
I20260807 18:18:40.944787 19249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:40.944206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.61:32933 (local address 127.0.0.1:36842)
0807 18:18:40.944548 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0807 18:18:40.944641 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.61:32933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185}
I20260807 18:18:40.945926 19249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:40.945591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.61:32933 (local address 127.0.0.1:36848)
0807 18:18:40.945721 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0807 18:18:40.945808 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.61:32933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260807 18:18:40.946175 19245 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.154.61:32933: Network error: Client connection negotiation failed: client connection to 127.18.154.61:32933: connect: Connection refused (error 111)
W20260807 18:18:40.983433 19254 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 18:18:40.983815 19254 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:18:40.983952 19254 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:18:40.984105 19254 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:18:40.991727 19254 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:18:40.991823 19254 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:18:40.991930 19254 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:18:40.992017 19254 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:18:40.992107 19254 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:18:41.002758 19254 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.154.20:35749
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126328808105-19048-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126328808105-19048-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.18.154.62:33241,127.18.154.61:32933,127.18.154.60:36053
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.61.keytab
--principal=kudu/127.18.154.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.154.61:32933
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126328808105-19048-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.18.154.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126328808105-19048-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:18:41.004194 19254 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:18:41.005600 19254 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:18:41.018203 19261 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:18:41.018407 19264 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:18:41.018420 19260 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:18:41.019393 19254 server_base.cc:1034] running on GCE node
I20260807 18:18:41.020288 19254 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:18:41.021924 19254 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:18:41.023240 19254 hybrid_clock.cc:648] HybridClock initialized: now 1786126721023149 us; error 58 us; skew 500 ppm
I20260807 18:18:41.023628 19254 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:18:41 dist-test-slave-gkw7 krb5kdc[19132](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126721, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:18:41.027073 19254 init.cc:377] Logged in from keytab as kudu/127.18.154.61@KRBTEST.COM (short username kudu)
I20260807 18:18:41.028683 19254 webserver.cc:460] Webserver started at http://127.18.154.61:45905/ using document root <none> and password file <none>
I20260807 18:18:41.029439 19254 fs_manager.cc:359] Metadata directory not provided
I20260807 18:18:41.029597 19254 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:18:41.029959 19254 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:18:41.033088 19254 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126328808105-19048-0/minicluster-data/master-1/data/instance:
uuid: "aa90e4d477fe4002b28f322d79605f29"
format_stamp: "Formatted at 2026-08-07 18:18:41 on dist-test-slave-gkw7"
server_key: "bfac04a4fbde2692b2e7b1e8e4482ade"
server_key_iv: "c18fb67be6ce7651dce4351e2552b078"
server_key_version: "encryptionkey@0"
I20260807 18:18:41.033995 19254 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126328808105-19048-0/minicluster-data/master-1/wal/instance:
uuid: "aa90e4d477fe4002b28f322d79605f29"
format_stamp: "Formatted at 2026-08-07 18:18:41 on dist-test-slave-gkw7"
server_key: "bfac04a4fbde2692b2e7b1e8e4482ade"
server_key_iv: "c18fb67be6ce7651dce4351e2552b078"
server_key_version: "encryptionkey@0"
I20260807 18:18:41.039477 19254 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260807 18:18:41.043119 19273 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:18:41.044487 19254 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 18:18:41.044795 19254 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126328808105-19048-0/minicluster-data/master-1/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126328808105-19048-0/minicluster-data/master-1/wal
uuid: "aa90e4d477fe4002b28f322d79605f29"
format_stamp: "Formatted at 2026-08-07 18:18:41 on dist-test-slave-gkw7"
server_key: "bfac04a4fbde2692b2e7b1e8e4482ade"
server_key_iv: "c18fb67be6ce7651dce4351e2552b078"
server_key_version: "encryptionkey@0"
I20260807 18:18:41.045035 19254 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126328808105-19048-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126328808105-19048-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126328808105-19048-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:18:41.074892 19254 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:18:41.079744 19254 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:18:41.080057 19254 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:18:41.099550 19254 rpc_server.cc:307] RPC server started. Bound to: 127.18.154.61:32933
I20260807 18:18:41.099557 19341 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.154.61:32933 every 8 connection(s)
I20260807 18:18:41.101058 19254 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126328808105-19048-0/minicluster-data/master-1/data/info.pb
I20260807 18:18:41.105088 19342 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:18:41.106024 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 19254
I20260807 18:18:41.106285 19048 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126328808105-19048-0/minicluster-data/master-1/wal/instance
I20260807 18:18:41.106748 19048 external_mini_cluster.cc:1428] Setting key 95862e8ed1f40cb898cd9bc2ce6200f4
I20260807 18:18:41.113144 19342 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.154.62" port: 33241 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 18:18:41 dist-test-slave-gkw7 krb5kdc[19132](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126721, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.61@KRBTEST.COM for kudu/127.18.154.62@KRBTEST.COM
I20260807 18:18:41.138981 19344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:41.116505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42752 (local address 127.18.154.62:33241)
0807 18:18:41.117996 (+ 1491us) server_negotiation.cc:206] Beginning negotiation
0807 18:18:41.118005 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:18:41.118496 (+ 491us) server_negotiation.cc:407] Connection header received
0807 18:18:41.120539 (+ 2043us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:18:41.120545 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:18:41.120645 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:18:41.120783 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:18:41.123187 (+ 2404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:41.123977 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:41.126606 (+ 2629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:41.127033 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:41.127142 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:18:41.132155 (+ 5013us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:18:41.132161 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:18:41.132165 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:18:41.132207 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:18:41.135196 (+ 2989us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:18:41.136296 (+ 1100us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:18:41.136301 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:18:41.136304 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:18:41.136432 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:18:41.136992 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:18:41.136996 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:18:41.136998 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:18:41.137303 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:18:41.137424 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0807 18:18:41.138501 (+ 1077us) server_negotiation.cc:299] Negotiation successful
0807 18:18:41.138703 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1324,"thread_start_us":1192,"threads_started":1}
I20260807 18:18:41.140832 19345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:41.117209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:33241 (local address 127.0.0.1:42752)
0807 18:18:41.118303 (+ 1094us) negotiation.cc:107] Waiting for socket to connect
0807 18:18:41.118344 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 18:18:41.120227 (+ 1883us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:18:41.121082 (+ 855us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:18:41.121096 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:18:41.122035 (+ 939us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:18:41.122927 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:18:41.122956 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:18:41.124167 (+ 1211us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:18:41.124176 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:18:41.126423 (+ 2247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:18:41.126442 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:18:41.127297 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:18:41.127307 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:18:41.127659 (+ 352us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:18:41.128528 (+ 869us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:18:41.128571 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:18:41.131832 (+ 3261us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:18:41.135382 (+ 3550us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:18:41.135398 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:18:41.135425 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:18:41.136155 (+ 730us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:18:41.136551 (+ 396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:18:41.136559 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:18:41.136565 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:18:41.136869 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:18:41.137447 (+ 578us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:18:41.137459 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:18:41.138019 (+ 560us) client_negotiation.cc:769] Sending connection context
0807 18:18:41.138378 (+ 359us) client_negotiation.cc:240] Negotiation successful
0807 18:18:41.139030 (+ 652us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":836,"thread_start_us":599,"threads_started":1}
I20260807 18:18:41.142930 19342 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.154.61" port: 32933 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.18.154.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.154.60@KRBTEST.COM" created.
Aug 07 18:18:41 dist-test-slave-gkw7 krb5kdc[19132](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126721, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.61@KRBTEST.COM for kudu/127.18.154.61@KRBTEST.COM
I20260807 18:18:41.158499 19345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:41.143906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.61:32933 (local address 127.0.0.1:36864)
0807 18:18:41.144118 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0807 18:18:41.144135 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 18:18:41.144295 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:18:41.145208 (+ 913us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:18:41.145212 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:18:41.145825 (+ 613us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:18:41.145976 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:18:41.145985 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:18:41.147559 (+ 1574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:18:41.147566 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:18:41.148498 (+ 932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:18:41.148510 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:18:41.149450 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:18:41.149455 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:18:41.149703 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:18:41.150374 (+ 671us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:18:41.150391 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:18:41.152434 (+ 2043us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:18:41.155871 (+ 3437us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:18:41.155879 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:18:41.155884 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:18:41.156258 (+ 374us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:18:41.156655 (+ 397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:18:41.156670 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:18:41.156673 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:18:41.156799 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:18:41.157441 (+ 642us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:18:41.157446 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:18:41.157612 (+ 166us) client_negotiation.cc:769] Sending connection context
0807 18:18:41.157774 (+ 162us) client_negotiation.cc:240] Negotiation successful
0807 18:18:41.158061 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":72,"mutex_wait_us":3}
I20260807 18:18:41.158541 19349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:41.144027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36864 (local address 127.18.154.61:32933)
0807 18:18:41.144778 (+ 751us) server_negotiation.cc:206] Beginning negotiation
0807 18:18:41.144784 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:18:41.144811 (+ 27us) server_negotiation.cc:407] Connection header received
0807 18:18:41.144859 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:18:41.144864 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:18:41.145055 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:18:41.145238 (+ 183us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:18:41.146167 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:41.147419 (+ 1252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:41.148646 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:41.149302 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:41.149437 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:18:41.152650 (+ 3213us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:18:41.152662 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:18:41.152678 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:18:41.152706 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:18:41.155729 (+ 3023us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:18:41.156413 (+ 684us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:18:41.156428 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:18:41.156431 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:18:41.156544 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:18:41.156904 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:18:41.156911 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:18:41.156915 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:18:41.157296 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:18:41.157518 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0807 18:18:41.157936 (+ 418us) server_negotiation.cc:299] Negotiation successful
0807 18:18:41.158166 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":46,"server-negotiator.queue_time_us":606,"thread_start_us":477,"threads_started":1}
I20260807 18:18:41.161760 19342 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.154.60" port: 36053 } has no permanent_uuid. Determining permanent_uuid...
I20260807 18:18:41.163300 19345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:41.162714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.60:36053 (local address 127.0.0.1:48154)
0807 18:18:41.162913 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0807 18:18:41.163085 (+ 172us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.60:36053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260807 18:18:41.163679 19275 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.154.60:36053: connect: Connection refused (error 111)
I20260807 18:18:41.165668 19345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:41.165133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.60:36053 (local address 127.0.0.1:48162)
0807 18:18:41.165341 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0807 18:18:41.165466 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.60:36053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260807 18:18:41.165947 19342 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.154.60:36053: Network error: Client connection negotiation failed: client connection to 127.18.154.60:36053: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.154.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.60.keytab.
Entry for principal kudu/127.18.154.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.60.keytab.
I20260807 18:18:41.189257 19048 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
/tmp/dist-test-taskTVw2hL/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126328808105-19048-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126328808105-19048-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126328808105-19048-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126328808105-19048-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.154.60:36053
--webserver_interface=127.18.154.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.60.keytab
--principal=kudu/127.18.154.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.154.20:35749
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.154.62:33241,127.18.154.61:32933,127.18.154.60:36053
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126328808105-19048-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126328808105-19048-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126328808105-19048-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 18:18:41.215335 19342 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.154.60" port: 36053 } attempt: 1
I20260807 18:18:41.216777 19345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:41.216150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.60:36053 (local address 127.0.0.1:48176)
0807 18:18:41.216408 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0807 18:18:41.216549 (+ 141us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.60:36053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260807 18:18:41.217979 19345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:41.217477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.60:36053 (local address 127.0.0.1:48192)
0807 18:18:41.217708 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0807 18:18:41.217817 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.60:36053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
W20260807 18:18:41.218235 19342 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.154.60:36053: Network error: Client connection negotiation failed: client connection to 127.18.154.60:36053: connect: Connection refused (error 111)
I20260807 18:18:41.245465 19245 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.154.61" port: 32933 } attempt: 5
Aug 07 18:18:41 dist-test-slave-gkw7 krb5kdc[19132](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126720, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.62@KRBTEST.COM for kudu/127.18.154.61@KRBTEST.COM
I20260807 18:18:41.259809 19349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:41.246442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36866 (local address 127.18.154.61:32933)
0807 18:18:41.246669 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0807 18:18:41.246678 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:18:41.246705 (+ 27us) server_negotiation.cc:407] Connection header received
0807 18:18:41.246946 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:18:41.246954 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:18:41.247027 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:18:41.247141 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:18:41.247967 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:41.248738 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:41.249972 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:41.250411 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:41.250545 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:18:41.254218 (+ 3673us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:18:41.254223 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:18:41.254226 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:18:41.254259 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:18:41.256936 (+ 2677us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:18:41.257806 (+ 870us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:18:41.257810 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:18:41.257812 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:18:41.257935 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:18:41.258407 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:18:41.258411 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:18:41.258413 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:18:41.258709 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:18:41.258821 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0807 18:18:41.259405 (+ 584us) server_negotiation.cc:299] Negotiation successful
0807 18:18:41.259589 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260807 18:18:41.260001 19249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:41.246373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.61:32933 (local address 127.0.0.1:36866)
0807 18:18:41.246621 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0807 18:18:41.246647 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0807 18:18:41.246798 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:18:41.247184 (+ 386us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:18:41.247192 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:18:41.247616 (+ 424us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:18:41.247809 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:18:41.247818 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:18:41.248881 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:18:41.248889 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:18:41.249827 (+ 938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:18:41.249840 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:18:41.250570 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:18:41.250576 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:18:41.250800 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:18:41.251451 (+ 651us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:18:41.251470 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:18:41.254006 (+ 2536us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:18:41.257077 (+ 3071us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:18:41.257085 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:18:41.257090 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:18:41.257686 (+ 596us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:18:41.258062 (+ 376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:18:41.258068 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:18:41.258072 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:18:41.258274 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:18:41.258841 (+ 567us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:18:41.258846 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:18:41.259050 (+ 204us) client_negotiation.cc:769] Sending connection context
0807 18:18:41.259265 (+ 215us) client_negotiation.cc:240] Negotiation successful
0807 18:18:41.259609 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":95}
I20260807 18:18:41.261328 19245 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.154.60" port: 36053 } has no permanent_uuid. Determining permanent_uuid...
I20260807 18:18:41.262755 19249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:41.262243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.60:36053 (local address 127.0.0.1:48194)
0807 18:18:41.262448 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0807 18:18:41.262565 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.60:36053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260807 18:18:41.263875 19249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:41.263397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.60:36053 (local address 127.0.0.1:48200)
0807 18:18:41.263582 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0807 18:18:41.263724 (+ 142us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.60:36053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260807 18:18:41.264140 19245 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.154.60:36053: Network error: Client connection negotiation failed: client connection to 127.18.154.60:36053: connect: Connection refused (error 111)
I20260807 18:18:41.286478 19342 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.154.60" port: 36053 } attempt: 2
I20260807 18:18:41.287945 19345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:41.287279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.60:36053 (local address 127.0.0.1:48206)
0807 18:18:41.287581 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0807 18:18:41.287712 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.60:36053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107}
I20260807 18:18:41.289156 19345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:41.288681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.60:36053 (local address 127.0.0.1:48208)
0807 18:18:41.288885 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0807 18:18:41.288990 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.60:36053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260807 18:18:41.289438 19342 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.154.60:36053: Network error: Client connection negotiation failed: client connection to 127.18.154.60:36053: connect: Connection refused (error 111)
I20260807 18:18:41.315404 19245 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.154.60" port: 36053 } attempt: 1
I20260807 18:18:41.316730 19249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:41.316166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.60:36053 (local address 127.0.0.1:48216)
0807 18:18:41.316395 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0807 18:18:41.316508 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.60:36053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260807 18:18:41.317792 19249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:41.317402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.60:36053 (local address 127.0.0.1:48230)
0807 18:18:41.317581 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0807 18:18:41.317666 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.60:36053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260807 18:18:41.318007 19245 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.154.60:36053: Network error: Client connection negotiation failed: client connection to 127.18.154.60:36053: connect: Connection refused (error 111)
I20260807 18:18:41.380757 19342 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.154.60" port: 36053 } attempt: 3
I20260807 18:18:41.382139 19345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:41.381516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.60:36053 (local address 127.0.0.1:48240)
0807 18:18:41.381822 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0807 18:18:41.381936 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.60:36053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260807 18:18:41.383389 19345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:41.382888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.60:36053 (local address 127.0.0.1:48252)
0807 18:18:41.383145 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0807 18:18:41.383246 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.60:36053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151}
W20260807 18:18:41.383673 19342 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.154.60:36053: Network error: Client connection negotiation failed: client connection to 127.18.154.60:36053: connect: Connection refused (error 111)
I20260807 18:18:41.386235 19245 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.154.60" port: 36053 } attempt: 2
I20260807 18:18:41.387346 19249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:41.386825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.60:36053 (local address 127.0.0.1:48262)
0807 18:18:41.387035 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0807 18:18:41.387151 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.60:36053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260807 18:18:41.388512 19249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:41.388121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.60:36053 (local address 127.0.0.1:48278)
0807 18:18:41.388296 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0807 18:18:41.388381 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.60:36053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260807 18:18:41.388779 19245 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.154.60:36053: Network error: Client connection negotiation failed: client connection to 127.18.154.60:36053: connect: Connection refused (error 111)
I20260807 18:18:41.495042 19245 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.154.60" port: 36053 } attempt: 3
I20260807 18:18:41.496364 19249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:41.495752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.60:36053 (local address 127.0.0.1:48284)
0807 18:18:41.496006 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0807 18:18:41.496127 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.60:36053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260807 18:18:41.497781 19249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:41.497233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.60:36053 (local address 127.0.0.1:48288)
0807 18:18:41.497494 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0807 18:18:41.497632 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.60:36053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120}
W20260807 18:18:41.498075 19245 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.154.60:36053: Network error: Client connection negotiation failed: client connection to 127.18.154.60:36053: connect: Connection refused (error 111)
I20260807 18:18:41.526916 19342 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.154.60" port: 36053 } attempt: 4
I20260807 18:18:41.528175 19345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:41.527588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.60:36053 (local address 127.0.0.1:48290)
0807 18:18:41.527862 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0807 18:18:41.527972 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.60:36053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
I20260807 18:18:41.529282 19345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:41.528882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.60:36053 (local address 127.0.0.1:48296)
0807 18:18:41.529051 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0807 18:18:41.529142 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.154.60:36053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260807 18:18:41.529711 19342 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.154.60:36053: Network error: Client connection negotiation failed: client connection to 127.18.154.60:36053: connect: Connection refused (error 111)
W20260807 18:18:41.562144 19353 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 18:18:41.562484 19353 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:18:41.562551 19353 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:18:41.562633 19353 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:18:41.570461 19353 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:18:41.570633 19353 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:18:41.570739 19353 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:18:41.570835 19353 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:18:41.570932 19353 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:18:41.581010 19353 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.154.20:35749
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126328808105-19048-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126328808105-19048-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.18.154.62:33241,127.18.154.61:32933,127.18.154.60:36053
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126328808105-19048-0/krb5kdc/kudu_127.18.154.60.keytab
--principal=kudu/127.18.154.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.154.60:36053
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126328808105-19048-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.18.154.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126328808105-19048-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:18:41.582521 19353 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:18:41.583956 19353 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:18:41.596486 19360 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:18:41.596575 19364 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:18:41.596530 19359 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:18:41.597417 19353 server_base.cc:1034] running on GCE node
I20260807 18:18:41.598345 19353 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:18:41.599854 19353 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:18:41.601136 19353 hybrid_clock.cc:648] HybridClock initialized: now 1786126721601045 us; error 58 us; skew 500 ppm
I20260807 18:18:41.601584 19353 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:18:41 dist-test-slave-gkw7 krb5kdc[19132](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126721, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:18:41.605187 19353 init.cc:377] Logged in from keytab as kudu/127.18.154.60@KRBTEST.COM (short username kudu)
I20260807 18:18:41.606901 19353 webserver.cc:460] Webserver started at http://127.18.154.60:36251/ using document root <none> and password file <none>
I20260807 18:18:41.607725 19353 fs_manager.cc:359] Metadata directory not provided
I20260807 18:18:41.607874 19353 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:18:41.608234 19353 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:18:41.611801 19353 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126328808105-19048-0/minicluster-data/master-2/data/instance:
uuid: "190a3adba0b24c4f91e507493888bb51"
format_stamp: "Formatted at 2026-08-07 18:18:41 on dist-test-slave-gkw7"
server_key: "8a98934c27eddd494a187724bb7466a9"
server_key_iv: "621dfa02ac5fcf60d579501ba2c3eb50"
server_key_version: "encryptionkey@0"
I20260807 18:18:41.612744 19353 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126328808105-19048-0/minicluster-data/master-2/wal/instance:
uuid: "190a3adba0b24c4f91e507493888bb51"
format_stamp: "Formatted at 2026-08-07 18:18:41 on dist-test-slave-gkw7"
server_key: "8a98934c27eddd494a187724bb7466a9"
server_key_iv: "621dfa02ac5fcf60d579501ba2c3eb50"
server_key_version: "encryptionkey@0"
I20260807 18:18:41.618580 19353 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260807 18:18:41.622454 19374 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:18:41.623909 19353 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260807 18:18:41.624109 19353 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126328808105-19048-0/minicluster-data/master-2/data,/tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126328808105-19048-0/minicluster-data/master-2/wal
uuid: "190a3adba0b24c4f91e507493888bb51"
format_stamp: "Formatted at 2026-08-07 18:18:41 on dist-test-slave-gkw7"
server_key: "8a98934c27eddd494a187724bb7466a9"
server_key_iv: "621dfa02ac5fcf60d579501ba2c3eb50"
server_key_version: "encryptionkey@0"
I20260807 18:18:41.624347 19353 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126328808105-19048-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126328808105-19048-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126328808105-19048-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:18:41.637753 19353 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:18:41.642359 19353 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:18:41.642657 19353 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:18:41.662341 19353 rpc_server.cc:307] RPC server started. Bound to: 127.18.154.60:36053
I20260807 18:18:41.662362 19454 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.154.60:36053 every 8 connection(s)
I20260807 18:18:41.663818 19353 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126328808105-19048-0/minicluster-data/master-2/data/info.pb
I20260807 18:18:41.664978 19048 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu as pid 19353
I20260807 18:18:41.665207 19048 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTVw2hL/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786126328808105-19048-0/minicluster-data/master-2/wal/instance
I20260807 18:18:41.665725 19048 external_mini_cluster.cc:1428] Setting key a0b2b9660dc7f76360325d0e915e4c83
I20260807 18:18:41.668713 19456 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:18:41.675319 19245 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.154.60" port: 36053 } attempt: 4
I20260807 18:18:41.678015 19456 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.154.62" port: 33241 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 18:18:41 dist-test-slave-gkw7 krb5kdc[19132](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126719, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.154.60@KRBTEST.COM
Aug 07 18:18:41 dist-test-slave-gkw7 krb5kdc[19132](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126720, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.62@KRBTEST.COM for kudu/127.18.154.60@KRBTEST.COM
I20260807 18:18:41.693472 19462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:41.667786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48310 (local address 127.18.154.60:36053)
0807 18:18:41.668562 (+ 776us) server_negotiation.cc:206] Beginning negotiation
0807 18:18:41.668576 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 18:18:41.668619 (+ 43us) server_negotiation.cc:407] Connection header received
0807 18:18:41.670369 (+ 1750us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:18:41.670392 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:18:41.670794 (+ 402us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:18:41.671124 (+ 330us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:18:41.672917 (+ 1793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:41.674411 (+ 1494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:41.676354 (+ 1943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:41.677662 (+ 1308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:41.678525 (+ 863us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:18:41.683329 (+ 4804us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:18:41.683345 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:18:41.683365 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:18:41.683421 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:18:41.688183 (+ 4762us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:18:41.689151 (+ 968us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:18:41.689162 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:18:41.689171 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:18:41.689404 (+ 233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:18:41.689932 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:18:41.689938 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:18:41.689941 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:18:41.690580 (+ 639us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:18:41.690953 (+ 373us) server_negotiation.cc:1035] Waiting for connection context
0807 18:18:41.691415 (+ 462us) server_negotiation.cc:299] Negotiation successful
0807 18:18:41.692070 (+ 655us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":127,"server-negotiator.queue_time_us":562,"spinlock_wait_cycles":15104,"thread_start_us":331,"threads_started":1}
I20260807 18:18:41.693712 19249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:41.676003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.60:36053 (local address 127.0.0.1:48312)
0807 18:18:41.676365 (+ 362us) negotiation.cc:107] Waiting for socket to connect
0807 18:18:41.676392 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0807 18:18:41.676536 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:18:41.677707 (+ 1171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:18:41.677714 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:18:41.678180 (+ 466us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:18:41.678330 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:18:41.678339 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:18:41.679868 (+ 1529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:18:41.679874 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:18:41.680811 (+ 937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:18:41.680831 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:18:41.682010 (+ 1179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:18:41.682016 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:18:41.682231 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:18:41.682947 (+ 716us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:18:41.682970 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:18:41.686054 (+ 3084us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:18:41.690674 (+ 4620us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:18:41.690681 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:18:41.690687 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:18:41.691274 (+ 587us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:18:41.691754 (+ 480us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:18:41.691758 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:18:41.691762 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:18:41.691973 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:18:41.692619 (+ 646us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:18:41.692624 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:18:41.692820 (+ 196us) client_negotiation.cc:769] Sending connection context
0807 18:18:41.692988 (+ 168us) client_negotiation.cc:240] Negotiation successful
0807 18:18:41.693285 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177}
I20260807 18:18:41.694640 19463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:41.676014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48312 (local address 127.18.154.60:36053)
0807 18:18:41.677375 (+ 1361us) server_negotiation.cc:206] Beginning negotiation
0807 18:18:41.677379 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:18:41.677398 (+ 19us) server_negotiation.cc:407] Connection header received
0807 18:18:41.677434 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:18:41.677437 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:18:41.677489 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:18:41.677645 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:18:41.678589 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:41.679701 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:41.680991 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:41.681806 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:41.682569 (+ 763us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:18:41.686318 (+ 3749us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:18:41.686324 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:18:41.686329 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:18:41.686368 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:18:41.690530 (+ 4162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:18:41.691424 (+ 894us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:18:41.691428 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:18:41.691431 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:18:41.691614 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:18:41.692104 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:18:41.692109 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:18:41.692113 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:18:41.692458 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:18:41.692602 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0807 18:18:41.693998 (+ 1396us) server_negotiation.cc:299] Negotiation successful
0807 18:18:41.694305 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":103,"server-negotiator.queue_time_us":1225,"thread_start_us":824,"threads_started":1}
Aug 07 18:18:41 dist-test-slave-gkw7 krb5kdc[19132](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126721, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.60@KRBTEST.COM for kudu/127.18.154.62@KRBTEST.COM
I20260807 18:18:41.703634 19465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:41.681471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.62:33241 (local address 127.0.0.1:42766)
0807 18:18:41.684656 (+ 3185us) negotiation.cc:107] Waiting for socket to connect
0807 18:18:41.684700 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 18:18:41.684869 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:18:41.686232 (+ 1363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:18:41.686245 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:18:41.687960 (+ 1715us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:18:41.688417 (+ 457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:18:41.688433 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:18:41.689811 (+ 1378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:18:41.689817 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:18:41.691853 (+ 2036us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:18:41.691865 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:18:41.692901 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:18:41.692908 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:18:41.693106 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:18:41.693727 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:18:41.693750 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:18:41.696852 (+ 3102us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:18:41.700802 (+ 3950us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:18:41.700813 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:18:41.700821 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:18:41.701203 (+ 382us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:18:41.701619 (+ 416us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:18:41.701625 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:18:41.701628 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:18:41.701793 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:18:41.702520 (+ 727us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:18:41.702527 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:18:41.702699 (+ 172us) client_negotiation.cc:769] Sending connection context
0807 18:18:41.702886 (+ 187us) client_negotiation.cc:240] Negotiation successful
0807 18:18:41.703218 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2901,"thread_start_us":2730,"threads_started":1}
I20260807 18:18:41.703673 19464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:41.681384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42766 (local address 127.18.154.62:33241)
0807 18:18:41.685634 (+ 4250us) server_negotiation.cc:206] Beginning negotiation
0807 18:18:41.685642 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:18:41.685657 (+ 15us) server_negotiation.cc:407] Connection header received
0807 18:18:41.685696 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:18:41.685700 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:18:41.685762 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:18:41.685884 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:18:41.688617 (+ 2733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:41.689622 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:41.691995 (+ 2373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:41.692677 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:41.692846 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:18:41.697464 (+ 4618us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:18:41.697474 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:18:41.697482 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:18:41.697552 (+ 70us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:18:41.700645 (+ 3093us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:18:41.701335 (+ 690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:18:41.701343 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:18:41.701347 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:18:41.701481 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:18:41.701906 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:18:41.701912 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:18:41.701916 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:18:41.702390 (+ 474us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:18:41.702528 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0807 18:18:41.702990 (+ 462us) server_negotiation.cc:299] Negotiation successful
0807 18:18:41.703213 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4060,"thread_start_us":3880,"threads_started":1}
I20260807 18:18:41.703945 19245 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e82bc516d09f4f9aa49c6d18eb877f01: Bootstrap starting.
I20260807 18:18:41.705607 19456 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.154.61" port: 32933 } has no permanent_uuid. Determining permanent_uuid...
I20260807 18:18:41.709717 19245 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e82bc516d09f4f9aa49c6d18eb877f01: Neither blocks nor log segments found. Creating new log.
I20260807 18:18:41.711503 19245 log.cc:824] T 00000000000000000000000000000000 P e82bc516d09f4f9aa49c6d18eb877f01: Log is configured to *not* fsync() on all Append() calls
Aug 07 18:18:41 dist-test-slave-gkw7 krb5kdc[19132](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126719, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.154.62@KRBTEST.COM
Aug 07 18:18:41 dist-test-slave-gkw7 krb5kdc[19132](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126721, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.60@KRBTEST.COM for kudu/127.18.154.61@KRBTEST.COM
I20260807 18:18:41.716995 19245 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e82bc516d09f4f9aa49c6d18eb877f01: No bootstrap required, opened a new log
I20260807 18:18:41.722620 19465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:41.706481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.61:32933 (local address 127.0.0.1:36870)
0807 18:18:41.706690 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0807 18:18:41.706710 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 18:18:41.706812 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:18:41.707528 (+ 716us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:18:41.707533 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:18:41.708030 (+ 497us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:18:41.708203 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:18:41.708215 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:18:41.709658 (+ 1443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:18:41.709662 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:18:41.711562 (+ 1900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:18:41.711578 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:18:41.712944 (+ 1366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:18:41.712951 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:18:41.713155 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:18:41.713834 (+ 679us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:18:41.713859 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:18:41.716426 (+ 2567us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:18:41.719700 (+ 3274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:18:41.719706 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:18:41.719710 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:18:41.720186 (+ 476us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:18:41.720689 (+ 503us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:18:41.720693 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:18:41.720696 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:18:41.720898 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:18:41.721587 (+ 689us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:18:41.721593 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:18:41.721767 (+ 174us) client_negotiation.cc:769] Sending connection context
0807 18:18:41.721933 (+ 166us) client_negotiation.cc:240] Negotiation successful
0807 18:18:41.722233 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":73}
I20260807 18:18:41.722643 19349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:41.706910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36870 (local address 127.18.154.61:32933)
0807 18:18:41.707107 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0807 18:18:41.707113 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:18:41.707131 (+ 18us) server_negotiation.cc:407] Connection header received
0807 18:18:41.707192 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:18:41.707197 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:18:41.707273 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:18:41.707364 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:18:41.708388 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:41.709422 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:41.711834 (+ 2412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:41.712524 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:41.712642 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:18:41.716627 (+ 3985us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:18:41.716633 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:18:41.716637 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:18:41.716680 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:18:41.719552 (+ 2872us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:18:41.720341 (+ 789us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:18:41.720347 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:18:41.720350 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:18:41.720525 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:18:41.721021 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:18:41.721027 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:18:41.721030 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:18:41.721457 (+ 427us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:18:41.721632 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0807 18:18:41.722027 (+ 395us) server_negotiation.cc:299] Negotiation successful
0807 18:18:41.722246 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 18:18:41.723635 19456 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.154.60" port: 36053 } has no permanent_uuid. Determining permanent_uuid...
I20260807 18:18:41.724677 19464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:41.698030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42772 (local address 127.18.154.62:33241)
0807 18:18:41.703984 (+ 5954us) server_negotiation.cc:206] Beginning negotiation
0807 18:18:41.703990 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:18:41.704005 (+ 15us) server_negotiation.cc:407] Connection header received
0807 18:18:41.704050 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:18:41.704055 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:18:41.704148 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:18:41.706411 (+ 2263us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:18:41.706548 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:41.707670 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:41.709608 (+ 1938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:41.710546 (+ 938us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:41.711270 (+ 724us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:18:41.716150 (+ 4880us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:18:41.716157 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:18:41.716161 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:18:41.716197 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:18:41.719984 (+ 3787us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:18:41.720896 (+ 912us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:18:41.720902 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:18:41.720905 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:18:41.721053 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:18:41.721505 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:18:41.721515 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:18:41.721519 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:18:41.721994 (+ 475us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:18:41.722258 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0807 18:18:41.724009 (+ 1751us) server_negotiation.cc:299] Negotiation successful
0807 18:18:41.724256 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5858,"thread_start_us":16044,"threads_started":1}
I20260807 18:18:41.728855 19245 raft_consensus.cc:348] T 00000000000000000000000000000000 P e82bc516d09f4f9aa49c6d18eb877f01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e82bc516d09f4f9aa49c6d18eb877f01" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 33241 } } peers { permanent_uuid: "aa90e4d477fe4002b28f322d79605f29" member_type: VOTER last_known_addr { host: "127.18.154.61" port: 32933 } } peers { permanent_uuid: "190a3adba0b24c4f91e507493888bb51" member_type: VOTER last_known_addr { host: "127.18.154.60" port: 36053 } }
I20260807 18:18:41.729347 19245 raft_consensus.cc:729] T 00000000000000000000000000000000 P e82bc516d09f4f9aa49c6d18eb877f01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e82bc516d09f4f9aa49c6d18eb877f01, State: Initialized, Role: FOLLOWER
I20260807 18:18:41.729887 19245 consensus_queue.cc:260] T 00000000000000000000000000000000 P e82bc516d09f4f9aa49c6d18eb877f01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e82bc516d09f4f9aa49c6d18eb877f01" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 33241 } } peers { permanent_uuid: "aa90e4d477fe4002b28f322d79605f29" member_type: VOTER last_known_addr { host: "127.18.154.61" port: 32933 } } peers { permanent_uuid: "190a3adba0b24c4f91e507493888bb51" member_type: VOTER last_known_addr { host: "127.18.154.60" port: 36053 } }
I20260807 18:18:41.730861 19470 sys_catalog.cc:455] T 00000000000000000000000000000000 P e82bc516d09f4f9aa49c6d18eb877f01 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e82bc516d09f4f9aa49c6d18eb877f01" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 33241 } } peers { permanent_uuid: "aa90e4d477fe4002b28f322d79605f29" member_type: VOTER last_known_addr { host: "127.18.154.61" port: 32933 } } peers { permanent_uuid: "190a3adba0b24c4f91e507493888bb51" member_type: VOTER last_known_addr { host: "127.18.154.60" port: 36053 } } }
I20260807 18:18:41.731341 19470 sys_catalog.cc:458] T 00000000000000000000000000000000 P e82bc516d09f4f9aa49c6d18eb877f01 [sys.catalog]: This master's current role is: FOLLOWER
Aug 07 18:18:41 dist-test-slave-gkw7 krb5kdc[19132](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126721, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.60@KRBTEST.COM for kudu/127.18.154.60@KRBTEST.COM
I20260807 18:18:41.732429 19245 sys_catalog.cc:564] T 00000000000000000000000000000000 P e82bc516d09f4f9aa49c6d18eb877f01 [sys.catalog]: configured and running, proceeding with master startup.
Aug 07 18:18:41 dist-test-slave-gkw7 krb5kdc[19132](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126719, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.154.61@KRBTEST.COM
I20260807 18:18:41.740622 19463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:41.724786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48318 (local address 127.18.154.60:36053)
0807 18:18:41.724960 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0807 18:18:41.724964 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:18:41.724976 (+ 12us) server_negotiation.cc:407] Connection header received
0807 18:18:41.725008 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:18:41.725011 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:18:41.725060 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:18:41.725163 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:18:41.725948 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:41.726826 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:41.728390 (+ 1564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:41.728951 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:41.729063 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:18:41.733121 (+ 4058us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:18:41.733128 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:18:41.733132 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:18:41.733170 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:18:41.737121 (+ 3951us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:18:41.738049 (+ 928us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:18:41.738058 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:18:41.738064 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:18:41.738214 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:18:41.738696 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:18:41.738704 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:18:41.738710 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:18:41.739183 (+ 473us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:18:41.739361 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0807 18:18:41.739905 (+ 544us) server_negotiation.cc:299] Negotiation successful
0807 18:18:41.740198 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 18:18:41.741477 19465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:41.724506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.60:36053 (local address 127.0.0.1:48318)
0807 18:18:41.724673 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0807 18:18:41.724694 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 18:18:41.724790 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:18:41.725179 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:18:41.725183 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:18:41.725659 (+ 476us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:18:41.725817 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:18:41.725825 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:18:41.726952 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:18:41.726960 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:18:41.728216 (+ 1256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:18:41.728228 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:18:41.729294 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:18:41.729299 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:18:41.729478 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:18:41.730192 (+ 714us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:18:41.730215 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:18:41.732919 (+ 2704us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:18:41.737294 (+ 4375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:18:41.737298 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:18:41.737301 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:18:41.737898 (+ 597us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:18:41.738379 (+ 481us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:18:41.738386 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:18:41.738391 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:18:41.738548 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:18:41.739442 (+ 894us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:18:41.739449 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:18:41.739636 (+ 187us) client_negotiation.cc:769] Sending connection context
0807 18:18:41.740821 (+ 1185us) client_negotiation.cc:240] Negotiation successful
0807 18:18:41.741139 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55}
I20260807 18:18:41.746094 19349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:41.727120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36884 (local address 127.18.154.61:32933)
0807 18:18:41.727287 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0807 18:18:41.727293 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:18:41.727312 (+ 19us) server_negotiation.cc:407] Connection header received
0807 18:18:41.727462 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:18:41.727466 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:18:41.727540 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:18:41.727643 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:18:41.729275 (+ 1632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:41.730713 (+ 1438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:41.732326 (+ 1613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:41.732829 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:41.732972 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:18:41.738389 (+ 5417us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:18:41.738400 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:18:41.738407 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:18:41.738452 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:18:41.742021 (+ 3569us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:18:41.743011 (+ 990us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:18:41.743016 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:18:41.743020 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:18:41.743194 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:18:41.743691 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:18:41.743696 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:18:41.743699 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:18:41.744045 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:18:41.744307 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0807 18:18:41.745632 (+ 1325us) server_negotiation.cc:299] Negotiation successful
0807 18:18:41.745851 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260807 18:18:41.748625 19456 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 190a3adba0b24c4f91e507493888bb51: Bootstrap starting.
I20260807 18:18:41.751164 19048 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:18:41.753983 19456 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 190a3adba0b24c4f91e507493888bb51: Neither blocks nor log segments found. Creating new log.
I20260807 18:18:41.754562 19212 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "e82bc516d09f4f9aa49c6d18eb877f01"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42772
I20260807 18:18:41.755136 19212 raft_consensus.cc:482] T 00000000000000000000000000000000 P e82bc516d09f4f9aa49c6d18eb877f01 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 18:18:41.755403 19212 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e82bc516d09f4f9aa49c6d18eb877f01 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:18:41.747059 19150 master_runner.cc:186] Error getting master registration for 127.18.154.61:32933: OK, instance_id { permanent_uuid: "aa90e4d477fe4002b28f322d79605f29" instance_seqno: 1786126721093921 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 18:18:41.757710 19150 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 18:18:41.760293 19212 raft_consensus.cc:504] T 00000000000000000000000000000000 P e82bc516d09f4f9aa49c6d18eb877f01 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e82bc516d09f4f9aa49c6d18eb877f01" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 33241 } } peers { permanent_uuid: "aa90e4d477fe4002b28f322d79605f29" member_type: VOTER last_known_addr { host: "127.18.154.61" port: 32933 } } peers { permanent_uuid: "190a3adba0b24c4f91e507493888bb51" member_type: VOTER last_known_addr { host: "127.18.154.60" port: 36053 } }
I20260807 18:18:41.762326 19456 log.cc:824] T 00000000000000000000000000000000 P 190a3adba0b24c4f91e507493888bb51: Log is configured to *not* fsync() on all Append() calls
W20260807 18:18:41.763762 19484 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e82bc516d09f4f9aa49c6d18eb877f01: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 18:18:41.763929 19484 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 18:18:41.764302 19212 leader_election.cc:288] T 00000000000000000000000000000000 P e82bc516d09f4f9aa49c6d18eb877f01 [CANDIDATE]: Term 1 election: Requested vote from peers aa90e4d477fe4002b28f322d79605f29 (127.18.154.61:32933), 190a3adba0b24c4f91e507493888bb51 (127.18.154.60:36053)
W20260807 18:18:41.764720 19180 tablet.cc:2369] T 00000000000000000000000000000000 P e82bc516d09f4f9aa49c6d18eb877f01: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:18:41.767163 19456 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 190a3adba0b24c4f91e507493888bb51: No bootstrap required, opened a new log
I20260807 18:18:41.778285 19456 raft_consensus.cc:348] T 00000000000000000000000000000000 P 190a3adba0b24c4f91e507493888bb51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e82bc516d09f4f9aa49c6d18eb877f01" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 33241 } } peers { permanent_uuid: "aa90e4d477fe4002b28f322d79605f29" member_type: VOTER last_known_addr { host: "127.18.154.61" port: 32933 } } peers { permanent_uuid: "190a3adba0b24c4f91e507493888bb51" member_type: VOTER last_known_addr { host: "127.18.154.60" port: 36053 } }
I20260807 18:18:41.778854 19456 raft_consensus.cc:729] T 00000000000000000000000000000000 P 190a3adba0b24c4f91e507493888bb51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 190a3adba0b24c4f91e507493888bb51, State: Initialized, Role: FOLLOWER
I20260807 18:18:41.779462 19456 consensus_queue.cc:260] T 00000000000000000000000000000000 P 190a3adba0b24c4f91e507493888bb51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e82bc516d09f4f9aa49c6d18eb877f01" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 33241 } } peers { permanent_uuid: "aa90e4d477fe4002b28f322d79605f29" member_type: VOTER last_known_addr { host: "127.18.154.61" port: 32933 } } peers { permanent_uuid: "190a3adba0b24c4f91e507493888bb51" member_type: VOTER last_known_addr { host: "127.18.154.60" port: 36053 } }
I20260807 18:18:41.786552 19497 sys_catalog.cc:455] T 00000000000000000000000000000000 P 190a3adba0b24c4f91e507493888bb51 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e82bc516d09f4f9aa49c6d18eb877f01" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 33241 } } peers { permanent_uuid: "aa90e4d477fe4002b28f322d79605f29" member_type: VOTER last_known_addr { host: "127.18.154.61" port: 32933 } } peers { permanent_uuid: "190a3adba0b24c4f91e507493888bb51" member_type: VOTER last_known_addr { host: "127.18.154.60" port: 36053 } } }
I20260807 18:18:41.789043 19497 sys_catalog.cc:458] T 00000000000000000000000000000000 P 190a3adba0b24c4f91e507493888bb51 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 18:18:41.788897 19456 sys_catalog.cc:564] T 00000000000000000000000000000000 P 190a3adba0b24c4f91e507493888bb51 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:18:41.788961 19413 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e82bc516d09f4f9aa49c6d18eb877f01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "190a3adba0b24c4f91e507493888bb51"
I20260807 18:18:41.789565 19413 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 190a3adba0b24c4f91e507493888bb51 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:18:41.793152 19413 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 190a3adba0b24c4f91e507493888bb51 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e82bc516d09f4f9aa49c6d18eb877f01 in term 1.
I20260807 18:18:41.793956 19176 leader_election.cc:302] T 00000000000000000000000000000000 P e82bc516d09f4f9aa49c6d18eb877f01 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 190a3adba0b24c4f91e507493888bb51, e82bc516d09f4f9aa49c6d18eb877f01; no voters:
I20260807 18:18:41.794549 19470 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e82bc516d09f4f9aa49c6d18eb877f01 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:18:41.794627 19349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:41.772355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36890 (local address 127.18.154.61:32933)
0807 18:18:41.772545 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0807 18:18:41.772553 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:18:41.772572 (+ 19us) server_negotiation.cc:407] Connection header received
0807 18:18:41.772636 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:18:41.772641 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:18:41.772720 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:18:41.772824 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:18:41.775437 (+ 2613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:41.776774 (+ 1337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:41.779240 (+ 2466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:41.780093 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:41.780213 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:18:41.785059 (+ 4846us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:18:41.785071 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:18:41.785079 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:18:41.785137 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:18:41.789413 (+ 4276us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:18:41.790919 (+ 1506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:18:41.790924 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:18:41.790927 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:18:41.791087 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:18:41.791840 (+ 753us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:18:41.791848 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:18:41.791855 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:18:41.792276 (+ 421us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:18:41.792517 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0807 18:18:41.793674 (+ 1157us) server_negotiation.cc:299] Negotiation successful
0807 18:18:41.794100 (+ 426us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 18:18:41.810999 19470 raft_consensus.cc:686] T 00000000000000000000000000000000 P e82bc516d09f4f9aa49c6d18eb877f01 [term 1 LEADER]: Becoming Leader. State: Replica: e82bc516d09f4f9aa49c6d18eb877f01, State: Running, Role: LEADER
I20260807 18:18:41.811856 19470 consensus_queue.cc:237] T 00000000000000000000000000000000 P e82bc516d09f4f9aa49c6d18eb877f01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e82bc516d09f4f9aa49c6d18eb877f01" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 33241 } } peers { permanent_uuid: "aa90e4d477fe4002b28f322d79605f29" member_type: VOTER last_known_addr { host: "127.18.154.61" port: 32933 } } peers { permanent_uuid: "190a3adba0b24c4f91e507493888bb51" member_type: VOTER last_known_addr { host: "127.18.154.60" port: 36053 } }
I20260807 18:18:41.818692 19463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:41.772669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48324 (local address 127.18.154.60:36053)
0807 18:18:41.772853 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0807 18:18:41.772859 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:18:41.789515 (+ 16656us) server_negotiation.cc:407] Connection header received
0807 18:18:41.789871 (+ 356us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:18:41.789877 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:18:41.789997 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:18:41.790085 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:18:41.793491 (+ 3406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:41.794635 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:41.798887 (+ 4252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:41.799688 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:41.799836 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:18:41.803978 (+ 4142us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:18:41.803986 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:18:41.803991 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:18:41.804040 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:18:41.810409 (+ 6369us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:18:41.814680 (+ 4271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:18:41.814687 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:18:41.814690 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:18:41.815006 (+ 316us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:18:41.815815 (+ 809us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:18:41.815848 (+ 33us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:18:41.815852 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:18:41.816517 (+ 665us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:18:41.817078 (+ 561us) server_negotiation.cc:1035] Waiting for connection context
0807 18:18:41.817574 (+ 496us) server_negotiation.cc:299] Negotiation successful
0807 18:18:41.818011 (+ 437us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260807 18:18:41.818995 19464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:41.781081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42776 (local address 127.18.154.62:33241)
0807 18:18:41.781295 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0807 18:18:41.781305 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:18:41.793380 (+ 12075us) server_negotiation.cc:407] Connection header received
0807 18:18:41.795510 (+ 2130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:18:41.795519 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:18:41.795626 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:18:41.795710 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:18:41.800621 (+ 4911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:41.802246 (+ 1625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:41.805683 (+ 3437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:41.806470 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:41.806601 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:18:41.810032 (+ 3431us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:18:41.810041 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:18:41.810046 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:18:41.810096 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:18:41.814017 (+ 3921us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:18:41.815317 (+ 1300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:18:41.815324 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:18:41.815329 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:18:41.815492 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:18:41.815966 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:18:41.815974 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:18:41.815979 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:18:41.816401 (+ 422us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:18:41.816681 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0807 18:18:41.818320 (+ 1639us) server_negotiation.cc:299] Negotiation successful
0807 18:18:41.818585 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260807 18:18:41.824024 19500 sys_catalog.cc:455] T 00000000000000000000000000000000 P e82bc516d09f4f9aa49c6d18eb877f01 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e82bc516d09f4f9aa49c6d18eb877f01. Latest consensus state: current_term: 1 leader_uuid: "e82bc516d09f4f9aa49c6d18eb877f01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e82bc516d09f4f9aa49c6d18eb877f01" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 33241 } } peers { permanent_uuid: "aa90e4d477fe4002b28f322d79605f29" member_type: VOTER last_known_addr { host: "127.18.154.61" port: 32933 } } peers { permanent_uuid: "190a3adba0b24c4f91e507493888bb51" member_type: VOTER last_known_addr { host: "127.18.154.60" port: 36053 } } }
I20260807 18:18:41.824487 19500 sys_catalog.cc:458] T 00000000000000000000000000000000 P e82bc516d09f4f9aa49c6d18eb877f01 [sys.catalog]: This master's current role is: LEADER
I20260807 18:18:41.825440 19505 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:18:41.827775 19505 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:18:41.828938 19342 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.154.60" port: 36053 } attempt: 5
Aug 07 18:18:41 dist-test-slave-gkw7 krb5kdc[19132](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126721, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.154.61@KRBTEST.COM for kudu/127.18.154.60@KRBTEST.COM
I20260807 18:18:41.849722 19413 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 190a3adba0b24c4f91e507493888bb51 [term 1 FOLLOWER]: Refusing update from remote peer e82bc516d09f4f9aa49c6d18eb877f01: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:18:41.850687 19470 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e82bc516d09f4f9aa49c6d18eb877f01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "190a3adba0b24c4f91e507493888bb51" member_type: VOTER last_known_addr { host: "127.18.154.60" port: 36053 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:18:41.862555 19345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:41.829899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.154.60:36053 (local address 127.0.0.1:48328)
0807 18:18:41.830186 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0807 18:18:41.830207 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 18:18:41.830318 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:18:41.831001 (+ 683us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:18:41.831006 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:18:41.831555 (+ 549us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:18:41.831763 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:18:41.831775 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:18:41.833588 (+ 1813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:18:41.833597 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:18:41.835562 (+ 1965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:18:41.835584 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:18:41.840958 (+ 5374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:18:41.840965 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:18:41.841210 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:18:41.842055 (+ 845us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:18:41.842078 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:18:41.845788 (+ 3710us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:18:41.857757 (+ 11969us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:18:41.857764 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:18:41.857772 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:18:41.858515 (+ 743us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:18:41.859398 (+ 883us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:18:41.859407 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:18:41.859427 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:18:41.859638 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:18:41.861085 (+ 1447us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:18:41.861093 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:18:41.861334 (+ 241us) client_negotiation.cc:769] Sending connection context
0807 18:18:41.861496 (+ 162us) client_negotiation.cc:240] Negotiation successful
0807 18:18:41.861960 (+ 464us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153}
I20260807 18:18:41.869890 19463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:41.829761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48328 (local address 127.18.154.60:36053)
0807 18:18:41.830545 (+ 784us) server_negotiation.cc:206] Beginning negotiation
0807 18:18:41.830551 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:18:41.830568 (+ 17us) server_negotiation.cc:407] Connection header received
0807 18:18:41.830623 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:18:41.830628 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:18:41.830708 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:18:41.830809 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:18:41.831939 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:41.833283 (+ 1344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:41.835943 (+ 2660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:41.840691 (+ 4748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:41.845402 (+ 4711us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:18:41.846009 (+ 607us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:18:41.846017 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:18:41.846022 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:18:41.846084 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:18:41.857495 (+ 11411us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:18:41.858790 (+ 1295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:18:41.858798 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:18:41.858804 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:18:41.859153 (+ 349us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:18:41.859855 (+ 702us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:18:41.859895 (+ 40us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:18:41.859900 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:18:41.860849 (+ 949us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:18:41.867960 (+ 7111us) server_negotiation.cc:1035] Waiting for connection context
0807 18:18:41.868252 (+ 292us) server_negotiation.cc:299] Negotiation successful
0807 18:18:41.868889 (+ 637us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":660}
I20260807 18:18:41.872176 19497 sys_catalog.cc:455] T 00000000000000000000000000000000 P 190a3adba0b24c4f91e507493888bb51 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e82bc516d09f4f9aa49c6d18eb877f01. Latest consensus state: current_term: 1 leader_uuid: "e82bc516d09f4f9aa49c6d18eb877f01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e82bc516d09f4f9aa49c6d18eb877f01" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 33241 } } peers { permanent_uuid: "aa90e4d477fe4002b28f322d79605f29" member_type: VOTER last_known_addr { host: "127.18.154.61" port: 32933 } } peers { permanent_uuid: "190a3adba0b24c4f91e507493888bb51" member_type: VOTER last_known_addr { host: "127.18.154.60" port: 36053 } } }
I20260807 18:18:41.872484 19497 sys_catalog.cc:458] T 00000000000000000000000000000000 P 190a3adba0b24c4f91e507493888bb51 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 18:18:41.875301 19509 mvcc.cc:205] Tried to move back new op lower bound from 7315975052669214720 to 7315975052548292608. Current Snapshot: MvccSnapshot[applied={T|T < 7315975052669214720}]
I20260807 18:18:41.883359 19342 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa90e4d477fe4002b28f322d79605f29: Bootstrap starting.
I20260807 18:18:41.888212 19513 mvcc.cc:205] Tried to move back new op lower bound from 7315975052669214720 to 7315975052548292608. Current Snapshot: MvccSnapshot[applied={T|T < 7315975052669214720}]
I20260807 18:18:41.895447 19342 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P aa90e4d477fe4002b28f322d79605f29: Neither blocks nor log segments found. Creating new log.
I20260807 18:18:41.899746 19342 log.cc:824] T 00000000000000000000000000000000 P aa90e4d477fe4002b28f322d79605f29: Log is configured to *not* fsync() on all Append() calls
I20260807 18:18:41.900337 19500 sys_catalog.cc:455] T 00000000000000000000000000000000 P e82bc516d09f4f9aa49c6d18eb877f01 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e82bc516d09f4f9aa49c6d18eb877f01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e82bc516d09f4f9aa49c6d18eb877f01" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 33241 } } peers { permanent_uuid: "aa90e4d477fe4002b28f322d79605f29" member_type: VOTER last_known_addr { host: "127.18.154.61" port: 32933 } } peers { permanent_uuid: "190a3adba0b24c4f91e507493888bb51" member_type: VOTER last_known_addr { host: "127.18.154.60" port: 36053 } } }
I20260807 18:18:41.900643 19500 sys_catalog.cc:458] T 00000000000000000000000000000000 P e82bc516d09f4f9aa49c6d18eb877f01 [sys.catalog]: This master's current role is: LEADER
I20260807 18:18:41.905247 19505 catalog_manager.cc:1347] Generated new cluster ID: 6a9ac25fbbca48ef99e377c39dedce7d
I20260807 18:18:41.905391 19505 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:18:41.906348 19497 sys_catalog.cc:455] T 00000000000000000000000000000000 P 190a3adba0b24c4f91e507493888bb51 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e82bc516d09f4f9aa49c6d18eb877f01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e82bc516d09f4f9aa49c6d18eb877f01" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 33241 } } peers { permanent_uuid: "aa90e4d477fe4002b28f322d79605f29" member_type: VOTER last_known_addr { host: "127.18.154.61" port: 32933 } } peers { permanent_uuid: "190a3adba0b24c4f91e507493888bb51" member_type: VOTER last_known_addr { host: "127.18.154.60" port: 36053 } } }
I20260807 18:18:41.906617 19497 sys_catalog.cc:458] T 00000000000000000000000000000000 P 190a3adba0b24c4f91e507493888bb51 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 18:18:41.915839 19342 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa90e4d477fe4002b28f322d79605f29: No bootstrap required, opened a new log
I20260807 18:18:41.941493 19505 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:18:41.943800 19505 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:18:41.943457 19342 raft_consensus.cc:348] T 00000000000000000000000000000000 P aa90e4d477fe4002b28f322d79605f29 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e82bc516d09f4f9aa49c6d18eb877f01" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 33241 } } peers { permanent_uuid: "aa90e4d477fe4002b28f322d79605f29" member_type: VOTER last_known_addr { host: "127.18.154.61" port: 32933 } } peers { permanent_uuid: "190a3adba0b24c4f91e507493888bb51" member_type: VOTER last_known_addr { host: "127.18.154.60" port: 36053 } }
I20260807 18:18:41.944306 19342 raft_consensus.cc:729] T 00000000000000000000000000000000 P aa90e4d477fe4002b28f322d79605f29 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa90e4d477fe4002b28f322d79605f29, State: Initialized, Role: FOLLOWER
I20260807 18:18:41.945421 19342 consensus_queue.cc:260] T 00000000000000000000000000000000 P aa90e4d477fe4002b28f322d79605f29 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e82bc516d09f4f9aa49c6d18eb877f01" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 33241 } } peers { permanent_uuid: "aa90e4d477fe4002b28f322d79605f29" member_type: VOTER last_known_addr { host: "127.18.154.61" port: 32933 } } peers { permanent_uuid: "190a3adba0b24c4f91e507493888bb51" member_type: VOTER last_known_addr { host: "127.18.154.60" port: 36053 } }
I20260807 18:18:41.961010 19307 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e82bc516d09f4f9aa49c6d18eb877f01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "aa90e4d477fe4002b28f322d79605f29"
I20260807 18:18:41.961522 19307 raft_consensus.cc:3037] T 00000000000000000000000000000000 P aa90e4d477fe4002b28f322d79605f29 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:18:41.964785 19307 raft_consensus.cc:2454] T 00000000000000000000000000000000 P aa90e4d477fe4002b28f322d79605f29 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e82bc516d09f4f9aa49c6d18eb877f01 in term 1.
I20260807 18:18:41.965833 19305 raft_consensus.cc:1261] T 00000000000000000000000000000000 P aa90e4d477fe4002b28f322d79605f29 [term 1 FOLLOWER]: Refusing update from remote peer e82bc516d09f4f9aa49c6d18eb877f01: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:18:41.967370 19500 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e82bc516d09f4f9aa49c6d18eb877f01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa90e4d477fe4002b28f322d79605f29" member_type: VOTER last_known_addr { host: "127.18.154.61" port: 32933 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:18:41.974025 19342 sys_catalog.cc:564] T 00000000000000000000000000000000 P aa90e4d477fe4002b28f322d79605f29 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:18:41.980003 19505 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e82bc516d09f4f9aa49c6d18eb877f01: Generated new TSK 0
I20260807 18:18:41.981081 19505 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:18:41.981978 19353 master_runner.cc:186] Error getting master registration for 127.18.154.61:32933: OK, instance_id { permanent_uuid: "aa90e4d477fe4002b28f322d79605f29" instance_seqno: 1786126721093921 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 18:18:42.049866 19525 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa90e4d477fe4002b28f322d79605f29 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e82bc516d09f4f9aa49c6d18eb877f01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e82bc516d09f4f9aa49c6d18eb877f01" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 33241 } } peers { permanent_uuid: "aa90e4d477fe4002b28f322d79605f29" member_type: VOTER last_known_addr { host: "127.18.154.61" port: 32933 } } peers { permanent_uuid: "190a3adba0b24c4f91e507493888bb51" member_type: VOTER last_known_addr { host: "127.18.154.60" port: 36053 } } }
I20260807 18:18:42.050414 19525 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa90e4d477fe4002b28f322d79605f29 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 18:18:42.050199 19470 sys_catalog.cc:455] T 00000000000000000000000000000000 P e82bc516d09f4f9aa49c6d18eb877f01 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e82bc516d09f4f9aa49c6d18eb877f01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e82bc516d09f4f9aa49c6d18eb877f01" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 33241 } } peers { permanent_uuid: "aa90e4d477fe4002b28f322d79605f29" member_type: VOTER last_known_addr { host: "127.18.154.61" port: 32933 } } peers { permanent_uuid: "190a3adba0b24c4f91e507493888bb51" member_type: VOTER last_known_addr { host: "127.18.154.60" port: 36053 } } }
I20260807 18:18:42.050545 19470 sys_catalog.cc:458] T 00000000000000000000000000000000 P e82bc516d09f4f9aa49c6d18eb877f01 [sys.catalog]: This master's current role is: LEADER
I20260807 18:18:42.050940 19529 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa90e4d477fe4002b28f322d79605f29 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e82bc516d09f4f9aa49c6d18eb877f01. Latest consensus state: current_term: 1 leader_uuid: "e82bc516d09f4f9aa49c6d18eb877f01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e82bc516d09f4f9aa49c6d18eb877f01" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 33241 } } peers { permanent_uuid: "aa90e4d477fe4002b28f322d79605f29" member_type: VOTER last_known_addr { host: "127.18.154.61" port: 32933 } } peers { permanent_uuid: "190a3adba0b24c4f91e507493888bb51" member_type: VOTER last_known_addr { host: "127.18.154.60" port: 36053 } } }
I20260807 18:18:42.051471 19529 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa90e4d477fe4002b28f322d79605f29 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 18:18:42.053284 19537 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa90e4d477fe4002b28f322d79605f29 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e82bc516d09f4f9aa49c6d18eb877f01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e82bc516d09f4f9aa49c6d18eb877f01" member_type: VOTER last_known_addr { host: "127.18.154.62" port: 33241 } } peers { permanent_uuid: "aa90e4d477fe4002b28f322d79605f29" member_type: VOTER last_known_addr { host: "127.18.154.61" port: 32933 } } peers { permanent_uuid: "190a3adba0b24c4f91e507493888bb51" member_type: VOTER last_known_addr { host: "127.18.154.60" port: 36053 } } }
I20260807 18:18:42.053956 19537 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa90e4d477fe4002b28f322d79605f29 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 18:18:42.057734 19533 mvcc.cc:205] Tried to move back new op lower bound from 7315975053037670400 to 7315975052548292608. Current Snapshot: MvccSnapshot[applied={T|T < 7315975053037670400}]
I20260807 18:18:42.059135 19353 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 18:18:42.065347 19527 catalog_manager.cc:1259] Loaded cluster ID: 6a9ac25fbbca48ef99e377c39dedce7d
I20260807 18:18:42.065461 19527 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 190a3adba0b24c4f91e507493888bb51: loading cluster ID for follower catalog manager: success
I20260807 18:18:42.069610 19527 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 190a3adba0b24c4f91e507493888bb51: acquiring CA information for follower catalog manager: success
I20260807 18:18:42.072513 19527 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 190a3adba0b24c4f91e507493888bb51: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 18:18:42.112323 19551 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
W20260807 18:18:42.147574 19048 mini_kdc.cc:348] Time spent kdestroy: real 0.150s user 0.000s sys 0.037s
W20260807 18:18:42.171316 19464 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:18:42.155374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42792 (local address 127.18.154.62:33241)
0807 18:18:42.155620 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0807 18:18:42.155626 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:18:42.168036 (+ 12410us) server_negotiation.cc:407] Connection header received
0807 18:18:42.168838 (+ 802us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:18:42.168850 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:18:42.168982 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:18:42.169086 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:18:42.170955 (+ 1869us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42792: BlockingRecv error: recv got EOF from 127.0.0.1:42792 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
W20260807 18:18:42.174199 19349 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:18:42.155839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36900 (local address 127.18.154.61:32933)
0807 18:18:42.156070 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0807 18:18:42.156110 (+ 40us) server_negotiation.cc:399] Waiting for connection header
0807 18:18:42.167366 (+ 11256us) server_negotiation.cc:407] Connection header received
0807 18:18:42.172126 (+ 4760us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:18:42.172136 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:18:42.172260 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:18:42.172356 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:18:42.173908 (+ 1552us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36900: BlockingRecv error: recv got EOF from 127.0.0.1:36900 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
W20260807 18:18:42.177735 19463 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:18:42.160833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48332 (local address 127.18.154.60:36053)
0807 18:18:42.161043 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0807 18:18:42.161049 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:18:42.175387 (+ 14338us) server_negotiation.cc:407] Connection header received
0807 18:18:42.175933 (+ 546us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:18:42.175942 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:18:42.176053 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:18:42.176142 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:18:42.177352 (+ 1210us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48332: BlockingRecv error: recv got EOF from 127.0.0.1:48332 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
W20260807 18:18:42.185021 19349 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:18:42.182878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36910 (local address 127.18.154.61:32933)
0807 18:18:42.183097 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0807 18:18:42.183106 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:18:42.183590 (+ 484us) server_negotiation.cc:407] Connection header received
0807 18:18:42.183835 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:18:42.183839 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:18:42.183938 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:18:42.184049 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:18:42.184762 (+ 713us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36910: BlockingRecv error: recv got EOF from 127.0.0.1:36910 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
W20260807 18:18:42.190999 19349 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:18:42.189126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36922 (local address 127.18.154.61:32933)
0807 18:18:42.189335 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0807 18:18:42.189342 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:18:42.189806 (+ 464us) server_negotiation.cc:407] Connection header received
0807 18:18:42.190007 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:18:42.190011 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:18:42.190079 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:18:42.190180 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:18:42.190809 (+ 629us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36922: BlockingRecv error: recv got EOF from 127.0.0.1:36922 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
W20260807 18:18:42.198649 19349 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:18:42.196671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36928 (local address 127.18.154.61:32933)
0807 18:18:42.196853 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0807 18:18:42.196862 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:18:42.197356 (+ 494us) server_negotiation.cc:407] Connection header received
0807 18:18:42.197572 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:18:42.197577 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:18:42.197665 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:18:42.197768 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:18:42.198419 (+ 651us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36928: BlockingRecv error: recv got EOF from 127.0.0.1:36928 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
W20260807 18:18:42.207664 19349 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:18:42.205664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36932 (local address 127.18.154.61:32933)
0807 18:18:42.205916 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0807 18:18:42.205924 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:18:42.206404 (+ 480us) server_negotiation.cc:407] Connection header received
0807 18:18:42.206614 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:18:42.206619 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:18:42.206697 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:18:42.206800 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:18:42.207441 (+ 641us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36932: BlockingRecv error: recv got EOF from 127.0.0.1:36932 (error 108)
Metrics: {"server-negotiator.queue_time_us":110}
W20260807 18:18:42.220952 19349 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:18:42.218847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36946 (local address 127.18.154.61:32933)
0807 18:18:42.219071 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0807 18:18:42.219079 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:18:42.219607 (+ 528us) server_negotiation.cc:407] Connection header received
0807 18:18:42.219831 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:18:42.219843 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:18:42.219918 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:18:42.220026 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:18:42.220729 (+ 703us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36946: BlockingRecv error: recv got EOF from 127.0.0.1:36946 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
W20260807 18:18:42.245484 19349 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:18:42.243043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36962 (local address 127.18.154.61:32933)
0807 18:18:42.243238 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0807 18:18:42.243245 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:18:42.243805 (+ 560us) server_negotiation.cc:407] Connection header received
0807 18:18:42.244004 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:18:42.244011 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:18:42.244137 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:18:42.244277 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:18:42.245218 (+ 941us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36962: BlockingRecv error: recv got EOF from 127.0.0.1:36962 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
W20260807 18:18:42.284634 19349 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:18:42.281860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36966 (local address 127.18.154.61:32933)
0807 18:18:42.282081 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0807 18:18:42.282089 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:18:42.282701 (+ 612us) server_negotiation.cc:407] Connection header received
0807 18:18:42.282967 (+ 266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:18:42.282976 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:18:42.283099 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:18:42.283243 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:18:42.284292 (+ 1049us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36966: BlockingRecv error: recv got EOF from 127.0.0.1:36966 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
W20260807 18:18:42.367746 19349 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:18:42.365131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36982 (local address 127.18.154.61:32933)
0807 18:18:42.365335 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0807 18:18:42.365343 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:18:42.366020 (+ 677us) server_negotiation.cc:407] Connection header received
0807 18:18:42.366262 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:18:42.366269 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:18:42.366390 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:18:42.366542 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:18:42.367479 (+ 937us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36982: BlockingRecv error: recv got EOF from 127.0.0.1:36982 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
W20260807 18:18:42.502022 19349 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:18:42.499739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36992 (local address 127.18.154.61:32933)
0807 18:18:42.500016 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0807 18:18:42.500025 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:18:42.500357 (+ 332us) server_negotiation.cc:407] Connection header received
0807 18:18:42.500549 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:18:42.500558 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:18:42.500681 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:18:42.500829 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:18:42.501684 (+ 855us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36992: BlockingRecv error: recv got EOF from 127.0.0.1:36992 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
W20260807 18:18:42.765590 19349 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:18:42.763581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37008 (local address 127.18.154.61:32933)
0807 18:18:42.763838 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0807 18:18:42.763845 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:18:42.764019 (+ 174us) server_negotiation.cc:407] Connection header received
0807 18:18:42.764307 (+ 288us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:18:42.764311 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:18:42.764387 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:18:42.764505 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:18:42.765329 (+ 824us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37008: BlockingRecv error: recv got EOF from 127.0.0.1:37008 (error 108)
Metrics: {"server-negotiator.queue_time_us":99}
I20260807 18:18:43.117800 19551 catalog_manager.cc:1259] Loaded cluster ID: 6a9ac25fbbca48ef99e377c39dedce7d
I20260807 18:18:43.118008 19551 catalog_manager.cc:1552] T 00000000000000000000000000000000 P aa90e4d477fe4002b28f322d79605f29: loading cluster ID for follower catalog manager: success
I20260807 18:18:43.122073 19551 catalog_manager.cc:1574] T 00000000000000000000000000000000 P aa90e4d477fe4002b28f322d79605f29: acquiring CA information for follower catalog manager: success
I20260807 18:18:43.124739 19551 catalog_manager.cc:1602] T 00000000000000000000000000000000 P aa90e4d477fe4002b28f322d79605f29: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 18:18:43.295912 19649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:43.287770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37016 (local address 127.18.154.61:32933)
0807 18:18:43.288368 (+ 598us) server_negotiation.cc:206] Beginning negotiation
0807 18:18:43.288375 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:18:43.288783 (+ 408us) server_negotiation.cc:407] Connection header received
0807 18:18:43.288974 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:18:43.288980 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:18:43.289206 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:18:43.289361 (+ 155us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:18:43.289882 (+ 521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:43.291189 (+ 1307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:43.292706 (+ 1517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:43.293420 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:43.293630 (+ 210us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:18:43.294294 (+ 664us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:18:43.295110 (+ 816us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:18:43.295299 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0807 18:18:43.295451 (+ 152us) server_negotiation.cc:299] Negotiation successful
0807 18:18:43.295539 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":461,"thread_start_us":300,"threads_started":1}
I20260807 18:18:43.310623 19655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:43.303377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48344 (local address 127.18.154.60:36053)
0807 18:18:43.304117 (+ 740us) server_negotiation.cc:206] Beginning negotiation
0807 18:18:43.304123 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:18:43.304141 (+ 18us) server_negotiation.cc:407] Connection header received
0807 18:18:43.304239 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:18:43.304243 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:18:43.304404 (+ 161us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:18:43.304562 (+ 158us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:18:43.305122 (+ 560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:43.306054 (+ 932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:43.307921 (+ 1867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:43.308379 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:43.308482 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:18:43.309134 (+ 652us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:18:43.309812 (+ 678us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:18:43.309951 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0807 18:18:43.310130 (+ 179us) server_negotiation.cc:299] Negotiation successful
0807 18:18:43.310264 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":620,"thread_start_us":497,"threads_started":1}
I20260807 18:18:43.336891 19649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:43.317708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37026 (local address 127.18.154.61:32933)
0807 18:18:43.317896 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0807 18:18:43.317902 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:18:43.328572 (+ 10670us) server_negotiation.cc:407] Connection header received
0807 18:18:43.329206 (+ 634us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:18:43.329213 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:18:43.329309 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:18:43.329375 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:18:43.330163 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:43.331404 (+ 1241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:43.334403 (+ 2999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:43.335047 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:43.335181 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:18:43.335978 (+ 797us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:18:43.336137 (+ 159us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:18:43.336217 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0807 18:18:43.336449 (+ 232us) server_negotiation.cc:299] Negotiation successful
0807 18:18:43.336540 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 18:18:43.339644 19655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:43.317884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48360 (local address 127.18.154.60:36053)
0807 18:18:43.318109 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0807 18:18:43.318117 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:18:43.332780 (+ 14663us) server_negotiation.cc:407] Connection header received
0807 18:18:43.332996 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:18:43.333004 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:18:43.333118 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:18:43.333219 (+ 101us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:18:43.334177 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:43.335293 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:43.337414 (+ 2121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:43.338059 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:43.338170 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:18:43.338852 (+ 682us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:18:43.338977 (+ 125us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:18:43.339043 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0807 18:18:43.339261 (+ 218us) server_negotiation.cc:299] Negotiation successful
0807 18:18:43.339340 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260807 18:18:43.342367 19662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:18:43.317231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42796 (local address 127.18.154.62:33241)
0807 18:18:43.332617 (+ 15386us) server_negotiation.cc:206] Beginning negotiation
0807 18:18:43.332624 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:18:43.334409 (+ 1785us) server_negotiation.cc:407] Connection header received
0807 18:18:43.334542 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:18:43.334548 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:18:43.334763 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:18:43.334958 (+ 195us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:18:43.336069 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:43.337829 (+ 1760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:43.340052 (+ 2223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:18:43.340513 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:18:43.340664 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:18:43.341330 (+ 666us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:18:43.341756 (+ 426us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:18:43.341901 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0807 18:18:43.342011 (+ 110us) server_negotiation.cc:299] Negotiation successful
0807 18:18:43.342096 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15244,"thread_start_us":16641,"threads_started":1}
I20260807 18:18:43.355189 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 19150
I20260807 18:18:43.356534 19238 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:18:43.494876 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 19150
I20260807 18:18:43.525254 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 19254
I20260807 18:18:43.526633 19336 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:18:43.642277 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 19254
I20260807 18:18:43.669080 19048 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu pid 19353
I20260807 18:18:43.670440 19447 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:18:43.779743 19048 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTVw2hL/build/asan/bin/kudu with pid 19353
2026-08-07T18:18:43Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (5417 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (10745 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (394877 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST